<div class="medium-feed-item"><a href="https://python.plainenglish.io/10-python-projects-that-can-actually-make-you-money-43b05a96bf72?source=rss----78073def27b8---4"><img src="https://cdn-images-1.medium.com/max/2600/0*hXIuPzvK7nm65hgY" width="3984"></a>I Used to Build Python Projects Just to Practice. Then I Realized Some Projects Don&#x2019;t Just Teach You to Code &#x2014; They Can Become Real&#x2026;<a href="https://python.plainenglish.io/10-python-projects-that-can-actually-make-you-money-43b05a96bf72?source=rss----78073def27b8---4">Continue reading on Python in Plain English »</a></div>

I Used to Build Python Projects Just to Practice. Then I Realized Some Projects Don’t Just Teach You to Code — They Can Become Real…
Continue reading on Python in Plain English »

10 Python Projects That Can Actually Make You Money

<figure><img alt="How DORA Metrics Expose the Hidden Cost of Flaky Tests in Python CI Pipelines" src="https://cdn-images-1.medium.com/max/1024/1*sJZeOqEukFxMmsTX44eqTg.png" /></figure>Nobody on the team calls them a problem. That is the thing about flaky tests. They become part of the landscape so gradually that the team stops seeing them clearly. A test fails. Someone reruns the pipeline. It passes. The deployment proceeds. The failure gets attributed to a network hiccup, a timing issue, a transient dependency state. The team moves on.This happens three times a week. Then five. Then it is just how CI works on this project.The cost of this normalization is invisible in most of the metrics engineering teams track. Velocity stays the same. Sprint completion rates look fine. The pipeline is green often enough that nobody is raising an escalation. The flaky tests are a nuisance, not a crisis, which means they never make it to the top of anyone’s priority list.DORA metrics surface them differently. Not as a nuisance but as a measurable cost- one that shows up in the specific numbers that describe how reliably and efficiently a team delivers software. For Python teams running CI pipelines where flaky tests have become normalized background noise, tracking DORA metrics tends to produce a specific and uncomfortable realization: the background noise has been costing more than anyone estimated.<h3>What Flaky Tests Actually Are in Python Pipelines</h3>Before connecting flaky tests to <a href="https://keploy.io/blog/community/how-to-improve-dora-metrics">DORA metrics</a>, it helps to be precise about what causes test flakiness in Python CI pipelines specifically, because the causes determine where the cost shows up and how it can be addressed.The most common source of flaky tests in Python API and integration testing is non-deterministic response fields. When a test makes an API call and asserts against the full response, any field that changes between calls- timestamps, request IDs, UUIDs, session tokens, generated correlation identifiers- will cause the assertion to fail on subsequent runs even when the underlying behavior is correct. A test written against a response containing a timestamp field will pass when the timestamp matches the recorded value and fail when it does not, which is always on the next run.Python developers writing integration tests with frameworks like pytest often handle this manually- extracting the timestamp field from the assertion, writing custom comparison logic, or patching the time-dependent behavior in the test setup. Each of these approaches works for the specific field the developer thought to handle. Fields they did not think to handle keep causing failures.The second source is mock files that have drifted from current service behavior. A Python service that integrates with a downstream API records the expected response in a mock file and validates against it in CI. The downstream API deploys a change- a new required field, a restructured error response, a deprecated field removed, and the mock file becomes partially inaccurate. Tests that exercise the changed behavior fail. Tests that happen to exercise only unchanged behavior pass. The same test suite produces different results depending on which code paths get hit, which looks like flakiness but is actually the test suite accurately detecting that its assumptions are no longer universally correct.Both of these causes are structural rather than incidental. They do not get better through developer diligence alone. They require either systematic processes for keeping test assumptions current or tooling that handles the problem at the source.<h3>Where Flaky Tests Show Up in DORA Metrics</h3>DORA metrics do not have a “flaky test” category. They measure delivery outcomes rather than testing behaviors. But flaky tests leave specific fingerprints in DORA metrics that are recognizable once you know what to look for.Change failure rate is the first place flaky tests appear. When a test fails intermittently, and the team’s response is to rerun the pipeline until it passes, the underlying failure is not being investigated- it is being averaged away. Sometimes the failure is genuine and gets averaged past. When a deployment proceeds despite a genuine failure because the team assumed the failure was flakiness, the resulting production incident gets counted in the change failure rate. Teams with high levels of test flakiness tend to have elevated change failure rates partly because flakiness has trained them to discount test failures rather than investigate them.The second way flaky tests affect change failure rate is through the opposite behavior. Some teams respond to flakiness by investigating every failure, even the ones that resolve on rerun. This is the right instinct, but it consumes engineering time proportional to how often flaky failures occur. The time spent investigating flaky failures is time not spent on other work, and the rate of false investigations inflates the effective change failure rate because engineering capacity is being consumed by failures that did not represent genuine defects.Lead time for changes is the second DORA metric where flaky test costs are visible. A pipeline that requires manual reruns when flaky tests fail adds the rerun time to lead time for every affected deployment. If three percent of pipeline runs require a rerun due to flakiness, three percent of deployments take longer than they should. If ten percent require reruns, the impact on average lead time becomes significant at deployment frequency scale.The more significant lead time impact comes from teams that have added explicit retry logic to handle flakiness- configuring pytest to automatically retry failing tests a second or third time before reporting failure. This handles the symptom by adding execution time to every pipeline run, not just the ones affected by flakiness. A pipeline that retries each failing test twice takes longer on every run regardless of whether flakiness was present.Deployment rework rate captures the downstream cost of flaky tests that resulted in deployment holds, rollbacks, or post-deployment investigations. When a pipeline fails due to a flaky test and the failure is not recognized as flakiness in time, the deployment may be held pending investigation or rolled back as a precaution. Both actions show up in deployment rework rate because they consume deployment capacity without producing new value.<h3>What the Numbers Reveal That Developer Experience Does Not</h3>There is a specific disconnect between how Python development teams experience flaky tests and what DORA metrics reveal about their actual impact.The developer experience of flaky tests is characterized by small, forgettable friction. A test fails, the pipeline reruns, the day continues. No single incident is significant enough to warrant escalation or dedicated investigation time. The cumulative impact of dozens of these small friction events is never totalled because each one is individually forgettable.DORA metrics aggregate what developer experience fragments. A team experiencing five flaky test events per week, each requiring an average of twelve minutes of investigation and rerun time, is losing an hour of engineering capacity per week to flakiness. Over a quarter, that is thirteen hours- roughly a third of a full engineering week. That number does not appear anywhere until someone calculates it from the data that DORA metrics make visible.The change failure rate calculation is often more revealing. A team that has normalized rerunning flaky tests has implicitly accepted a policy of not investigating a certain percentage of test failures. When that percentage includes genuine failures, and it sometimes does, because the team has lost the ability to distinguish flaky failures from genuine ones at first glance- the change failure rate includes production incidents that were preventable. Attributing those incidents accurately to the policy of ignoring certain test failures requires the kind of systematic tracking that DORA metrics enable.<h3>What Causes Flakiness at the Python Integration Testing Layer</h3>Unit tests in Python can be flaky for various reasons- time-dependent assertions, random number generation, state pollution between tests. These are well-understood problems with established solutions in the Python ecosystem.Flakiness at the integration testing layer follows different patterns that are specific to how Python services interact with external dependencies.Python services making HTTP calls to downstream APIs are particularly susceptible to both forms of flakiness described earlier- non-deterministic response fields and mock drift. The combination of pytest as the testing framework and requests or httpx as the HTTP client creates a testing setup where the mock strategy has an outsized effect on how much flakiness the test suite experiences.Teams using responses or pytest-httpx to mock HTTP interactions write their mocks against a specific version of the downstream API’s behavior. When that behavior changes, the mocks become a source of either flakiness- if the change is partial and only some code paths are affected- or consistent failures. The distinction between a flaky test and a consistently failing test depends on which code paths a given test run exercises, which can vary based on test data, execution order, or the specific version of a dependency that happens to be installed.For Python API services specifically, Keploy addresses both sources of integration test flakiness at the layer where they originate. It captures real HTTP traffic between Python services and generates test cases and dependency mocks from those actual interactions rather than from manually written specifications. Critically, it runs multiple observations of each interaction to identify fields that vary across calls- timestamps, IDs, tokens, and automatically excludes those fields from assertions. The tests that result are stable across runs because the non-deterministic elements have been identified from real traffic patterns rather than through developer annotation.When downstream service behavior changes, new traffic captures reflect the updated behavior automatically. The drift between mock files and actual service behavior- the second major source of integration test flakiness in Python pipelines- does not accumulate because the source of the mock content is current observed behavior rather than a specification that ages between updates.<h3>Fixing the Problem vs Measuring the Cost</h3>The practical path for Python teams that have recognised flaky tests as a DORA metrics problem rather than just a developer experience problem has two components that work together rather than sequentially.Measuring the cost comes first, because it establishes the case for investment and identifies where flakiness is concentrated. Calculating lead time per deployment and separating pipeline runs that required reruns from those that did not gives a direct measure of flakiness overhead. Reviewing change failure rate investigations and tagging the ones that originated from test failures that were initially attributed to flakiness gives a measure of how often flakiness is masking genuine failures.Addressing the root causes comes second, because without structural changes the measurement effort produces data without improvement. The specific structural changes that address flakiness at the integration testing layer in Python pipelines are those that handle non-deterministic fields systematically rather than case by case, and that keep mock files current with actual service behavior rather than requiring manual updates after each downstream deployment.Teams that address these two root causes typically see change failure rate improve within one or two deployment cycles of making the change- not because their code quality improved, but because their test results became trustworthy enough to act on. The signal-to-noise ratio improved. Failures became meaningful rather than potentially ignorable. The team started investigating failures rather than rerunning them, which meant the failures that were genuine got caught rather than averaged past.The DORA metrics improvement follows from that behavioral change. Better signal leads to more reliable deployment decisions, which leads to a lower change failure rate and more predictable lead time. The flaky tests were not just a developer experience problem. They were a delivery performance problem that showed up in the numbers, and addressing them shows up in the numbers too.<h3>Before you go</h3><ul><li>Please take a moment to like the post and follow the writer!</li><li>Did you know that over 400,000 developers share what they’re building, learning, and discovering across our platforms every month? Learn how you can contribute <a href="https://plainenglish.io/write-for-us">here</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=19ac01b081cc" width="1" height="1" alt=""><hr><a href="https://python.plainenglish.io/how-dora-metrics-expose-the-hidden-cost-of-flaky-tests-in-python-ci-pipelines-19ac01b081cc">How DORA Metrics Expose the Hidden Cost of Flaky Tests in Python CI Pipelines</a> was originally published in <a href="https://python.plainenglish.io">Python in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

Nobody on the team calls them a problem. That is the thing about flaky tests. They become part of the landscape so gradually that the team stops seeing them clearly. A test fails. Someone reruns the pipeline. It passes. The deployment proceeds. The failure gets attributed to a network hiccup, a timing issue, a transient dependency state. The team moves on.
This happens three times a week. Then five. Then it is just how CI works on this project.
The cost of this normalization is invisible in most of the metrics engineering teams track. Velocity stays the same. Sprint completion rates look fine. The pipeline is green often enough that nobody is raising an escalation. The flaky tests are a nuisance, not a crisis, which means they never make it to the top of anyone’s priority list.
DORA metrics surface them differently. Not as a nuisance but as a measurable cost- one that shows up in the specific numbers that describe how reliably and efficiently a team delivers software. For Python teams running CI pipelines where flaky tests have become normalized background noise, tracking DORA metrics tends to produce a specific and uncomfortable realization: the background noise has been costing more than anyone estimated.
What Flaky Tests Actually Are in Python Pipelines
Before connecting flaky tests to DORA metrics, it helps to be precise about what causes test flakiness in Python CI pipelines specifically, because the causes determine where the cost shows up and how it can be addressed.
The most common source of flaky tests in Python API and integration testing is non-deterministic response fields. When a test makes an API call and asserts against the full response, any field that changes between calls- timestamps, request IDs, UUIDs, session tokens, generated correlation identifiers- will cause the assertion to fail on subsequent runs even when the underlying behavior is correct. A test written against a response containing a timestamp field will pass when the timestamp matches the recorded value and fail when it does not, which is always on the next run.
Python developers writing integration tests with frameworks like pytest often handle this manually- extracting the timestamp field from the assertion, writing custom comparison logic, or patching the time-dependent behavior in the test setup. Each of these approaches works for the specific field the developer thought to handle. Fields they did not think to handle keep causing failures.
The second source is mock files that have drifted from current service behavior. A Python service that integrates with a downstream API records the expected response in a mock file and validates against it in CI. The downstream API deploys a change- a new required field, a restructured error response, a deprecated field removed, and the mock file becomes partially inaccurate. Tests that exercise the changed behavior fail. Tests that happen to exercise only unchanged behavior pass. The same test suite produces different results depending on which code paths get hit, which looks like flakiness but is actually the test suite accurately detecting that its assumptions are no longer universally correct.
Both of these causes are structural rather than incidental. They do not get better through developer diligence alone. They require either systematic processes for keeping test assumptions current or tooling that handles the problem at the source.
Where Flaky Tests Show Up in DORA Metrics
DORA metrics do not have a “flaky test” category. They measure delivery outcomes rather than testing behaviors. But flaky tests leave specific fingerprints in DORA metrics that are recognizable once you know what to look for.
Change failure rate is the first place flaky tests appear. When a test fails intermittently, and the team’s response is to rerun the pipeline until it passes, the underlying failure is not being investigated- it is being averaged away. Sometimes the failure is genuine and gets averaged past. When a deployment proceeds despite a genuine failure because the team assumed the failure was flakiness, the resulting production incident gets counted in the change failure rate. Teams with high levels of test flakiness tend to have elevated change failure rates partly because flakiness has trained them to discount test failures rather than investigate them.
The second way flaky tests affect change failure rate is through the opposite behavior. Some teams respond to flakiness by investigating every failure, even the ones that resolve on rerun. This is the right instinct, but it consumes engineering time proportional to how often flaky failures occur. The time spent investigating flaky failures is time not spent on other work, and the rate of false investigations inflates the effective change failure rate because engineering capacity is being consumed by failures that did not represent genuine defects.
Lead time for changes is the second DORA metric where flaky test costs are visible. A pipeline that requires manual reruns when flaky tests fail adds the rerun time to lead time for every affected deployment. If three percent of pipeline runs require a rerun due to flakiness, three percent of deployments take longer than they should. If ten percent require reruns, the impact on average lead time becomes significant at deployment frequency scale.
The more significant lead time impact comes from teams that have added explicit retry logic to handle flakiness- configuring pytest to automatically retry failing tests a second or third time before reporting failure. This handles the symptom by adding execution time to every pipeline run, not just the ones affected by flakiness. A pipeline that retries each failing test twice takes longer on every run regardless of whether flakiness was present.
Deployment rework rate captures the downstream cost of flaky tests that resulted in deployment holds, rollbacks, or post-deployment investigations. When a pipeline fails due to a flaky test and the failure is not recognized as flakiness in time, the deployment may be held pending investigation or rolled back as a precaution. Both actions show up in deployment rework rate because they consume deployment capacity without producing new value.
What the Numbers Reveal That Developer Experience Does Not
There is a specific disconnect between how Python development teams experience flaky tests and what DORA metrics reveal about their actual impact.
The developer experience of flaky tests is characterized by small, forgettable friction. A test fails, the pipeline reruns, the day continues. No single incident is significant enough to warrant escalation or dedicated investigation time. The cumulative impact of dozens of these small friction events is never totalled because each one is individually forgettable.
DORA metrics aggregate what developer experience fragments. A team experiencing five flaky test events per week, each requiring an average of twelve minutes of investigation and rerun time, is losing an hour of engineering capacity per week to flakiness. Over a quarter, that is thirteen hours- roughly a third of a full engineering week. That number does not appear anywhere until someone calculates it from the data that DORA metrics make visible.
The change failure rate calculation is often more revealing. A team that has normalized rerunning flaky tests has implicitly accepted a policy of not investigating a certain percentage of test failures. When that percentage includes genuine failures, and it sometimes does, because the team has lost the ability to distinguish flaky failures from genuine ones at first glance- the change failure rate includes production incidents that were preventable. Attributing those incidents accurately to the policy of ignoring certain test failures requires the kind of systematic tracking that DORA metrics enable.
What Causes Flakiness at the Python Integration Testing Layer
Unit tests in Python can be flaky for various reasons- time-dependent assertions, random number generation, state pollution between tests. These are well-understood problems with established solutions in the Python ecosystem.
Flakiness at the integration testing layer follows different patterns that are specific to how Python services interact with external dependencies.
Python services making HTTP calls to downstream APIs are particularly susceptible to both forms of flakiness described earlier- non-deterministic response fields and mock drift. The combination of pytest as the testing framework and requests or httpx as the HTTP client creates a testing setup where the mock strategy has an outsized effect on how much flakiness the test suite experiences.
Teams using responses or pytest-httpx to mock HTTP interactions write their mocks against a specific version of the downstream API’s behavior. When that behavior changes, the mocks become a source of either flakiness- if the change is partial and only some code paths are affected- or consistent failures. The distinction between a flaky test and a consistently failing test depends on which code paths a given test run exercises, which can vary based on test data, execution order, or the specific version of a dependency that happens to be installed.
For Python API services specifically, Keploy addresses both sources of integration test flakiness at the layer where they originate. It captures real HTTP traffic between Python services and generates test cases and dependency mocks from those actual interactions rather than from manually written specifications. Critically, it runs multiple observations of each interaction to identify fields that vary across calls- timestamps, IDs, tokens, and automatically excludes those fields from assertions. The tests that result are stable across runs because the non-deterministic elements have been identified from real traffic patterns rather than through developer annotation.
When downstream service behavior changes, new traffic captures reflect the updated behavior automatically. The drift between mock files and actual service behavior- the second major source of integration test flakiness in Python pipelines- does not accumulate because the source of the mock content is current observed behavior rather than a specification that ages between updates.
Fixing the Problem vs Measuring the Cost
The practical path for Python teams that have recognised flaky tests as a DORA metrics problem rather than just a developer experience problem has two components that work together rather than sequentially.
Measuring the cost comes first, because it establishes the case for investment and identifies where flakiness is concentrated. Calculating lead time per deployment and separating pipeline runs that required reruns from those that did not gives a direct measure of flakiness overhead. Reviewing change failure rate investigations and tagging the ones that originated from test failures that were initially attributed to flakiness gives a measure of how often flakiness is masking genuine failures.
Addressing the root causes comes second, because without structural changes the measurement effort produces data without improvement. The specific structural changes that address flakiness at the integration testing layer in Python pipelines are those that handle non-deterministic fields systematically rather than case by case, and that keep mock files current with actual service behavior rather than requiring manual updates after each downstream deployment.
Teams that address these two root causes typically see change failure rate improve within one or two deployment cycles of making the change- not because their code quality improved, but because their test results became trustworthy enough to act on. The signal-to-noise ratio improved. Failures became meaningful rather than potentially ignorable. The team started investigating failures rather than rerunning them, which meant the failures that were genuine got caught rather than averaged past.
The DORA metrics improvement follows from that behavioral change. Better signal leads to more reliable deployment decisions, which leads to a lower change failure rate and more predictable lead time. The flaky tests were not just a developer experience problem. They were a delivery performance problem that showed up in the numbers, and addressing them shows up in the numbers too.
Before you go

Please take a moment to like the post and follow the writer!
Did you know that over 400,000 developers share what they’re building, learning, and discovering across our platforms every month? Learn how you can contribute here

How DORA Metrics Expose the Hidden Cost of Flaky Tests in Python CI Pipelines was originally published in Python in Plain English on Medium, where people are continuing the conversation by highlighting and responding to this story.

How DORA Metrics Expose the Hidden Cost of Flaky Tests in Python CI Pipelines

<div class="medium-feed-item"><a href="https://python.plainenglish.io/10-python-libraries-that-turn-difficult-problems-into-simple-solutions-c3b7aac0e511?source=rss----78073def27b8---4"><img src="https://cdn-images-1.medium.com/max/709/0*ALJaP7lRkmFK-F7V" width="709"></a>These Python tools remove hours of manual work by solving problems you probably shouldn&#x2019;t be solving yourself.<a href="https://python.plainenglish.io/10-python-libraries-that-turn-difficult-problems-into-simple-solutions-c3b7aac0e511?source=rss----78073def27b8---4">Continue reading on Python in Plain English »</a></div>

These Python tools remove hours of manual work by solving problems you probably shouldn’t be solving yourself.
Continue reading on Python in Plain English »

10 Python Libraries That Turn Difficult Problems Into Simple Solutions

<div class="medium-feed-item"><a href="https://python.plainenglish.io/10-github-repositories-to-master-python-in-2026-af1152e5d7ce?source=rss----78073def27b8---4"><img src="https://cdn-images-1.medium.com/max/1536/1*V4eYbJv8YvHw6PhMzQ8-AQ.png" width="1536"></a>Subtitle: I curated the best ones &#x2014; these will take you from intermediate to job-ready<a href="https://python.plainenglish.io/10-github-repositories-to-master-python-in-2026-af1152e5d7ce?source=rss----78073def27b8---4">Continue reading on Python in Plain English »</a></div>

Subtitle: I curated the best ones — these will take you from intermediate to job-ready
Continue reading on Python in Plain English »

10 GitHub Repositories to Master Python in 2026

<div class="medium-feed-item"><a href="https://python.plainenglish.io/10-python-tricks-that-make-your-code-instantly-more-elegant-5ffa44e4f660?source=rss----78073def27b8---4"><img src="https://cdn-images-1.medium.com/max/1536/0*ewqoMwJzFwCzYAVK" width="1536"></a>Small changes that eliminate boilerplate, reduce bugs, and make your Python code feel like it was written by someone with twice your&#x2026;<a href="https://python.plainenglish.io/10-python-tricks-that-make-your-code-instantly-more-elegant-5ffa44e4f660?source=rss----78073def27b8---4">Continue reading on Python in Plain English »</a></div>

Small changes that eliminate boilerplate, reduce bugs, and make your Python code feel like it was written by someone with twice your…
Continue reading on Python in Plain English »

10 Python Tricks That Make Your Code Instantly More Elegant

<div class="medium-feed-item"><a href="https://python.plainenglish.io/the-most-valuable-python-code-i-ever-wrote-was-code-i-never-shipped-10cb88a38b91?source=rss----78073def27b8---4"><img src="https://cdn-images-1.medium.com/max/1024/1*AMAWCtbXTRVfENRBVda5rg.jpeg" width="1024"></a>The scripts that quietly changed how I build software&#x200A;&#x2014;&#x200A;and why every Python developer should have a &#x201C;private toolbox.&#x201D;<a href="https://python.plainenglish.io/the-most-valuable-python-code-i-ever-wrote-was-code-i-never-shipped-10cb88a38b91?source=rss----78073def27b8---4">Continue reading on Python in Plain English »</a></div>

The scripts that quietly changed how I build software — and why every Python developer should have a “private toolbox.”
Continue reading on Python in Plain English »

The Most Valuable Python Code I Ever Wrote Was Code I Never Shipped

<div class="medium-feed-item"><a href="https://python.plainenglish.io/the-best-python-developers-i-know-rarely-start-with-python-3f3946fde3cc?source=rss----78073def27b8---4"><img src="https://cdn-images-1.medium.com/max/1024/1*m0AyiK847D7FVPKCqBH46Q.jpeg" width="1024"></a>The surprising habit I noticed after working through hundreds of Python projects &#x2014; and why it completely changed the way I solve problems.<a href="https://python.plainenglish.io/the-best-python-developers-i-know-rarely-start-with-python-3f3946fde3cc?source=rss----78073def27b8---4">Continue reading on Python in Plain English »</a></div>

The surprising habit I noticed after working through hundreds of Python projects — and why it completely changed the way I solve problems.
Continue reading on Python in Plain English »

The Best Python Developers I Know Rarely Start With Python

<div class="medium-feed-item"><a href="https://python.plainenglish.io/12-python-libraries-you-need-to-try-in-2026-7444b385044c?source=rss----78073def27b8---4"><img src="https://cdn-images-1.medium.com/max/1536/1*zIgsK1mRoFWfkWGBhGh4pw.png" width="1536"></a>MarkItDown, Polars, GPT Pilot, Smolagents, LangExtract, FastMCP &#x2014; I tested them all.<a href="https://python.plainenglish.io/12-python-libraries-you-need-to-try-in-2026-7444b385044c?source=rss----78073def27b8---4">Continue reading on Python in Plain English »</a></div>

MarkItDown, Polars, GPT Pilot, Smolagents, LangExtract, FastMCP — I tested them all.
Continue reading on Python in Plain English »

12 Python Libraries You Need to Try in 2026

<div class="medium-feed-item"><a href="https://python.plainenglish.io/10-python-projects-that-make-you-a-better-developer-d1584fc5569b?source=rss----78073def27b8---4"><img src="https://cdn-images-1.medium.com/max/680/0*mUlDqNbm7wrr9Shk" width="680"></a>Stop solving coding challenges alone. These hands-on projects teach the skills you&#x2019;ll actually use in production.<a href="https://python.plainenglish.io/10-python-projects-that-make-you-a-better-developer-d1584fc5569b?source=rss----78073def27b8---4">Continue reading on Python in Plain English »</a></div>

Stop solving coding challenges alone. These hands-on projects teach the skills you’ll actually use in production.
Continue reading on Python in Plain English »

10 Python Projects That Make You a Better Developer

<div class="medium-feed-item"><a href="https://python.plainenglish.io/agentic-automation-how-ai-agents-are-replacing-traditional-python-scripts-45feac26922f?source=rss----78073def27b8---4"><img src="https://cdn-images-1.medium.com/max/1536/1*YfZcFPrxlNeOhsqZiHg5bg.png" width="1536"></a>In 2026, &#x201C;writing a script&#x201D; is no longer the primary goal &#x2014; AI agents handle what scripts couldn&#x2019;t.<a href="https://python.plainenglish.io/agentic-automation-how-ai-agents-are-replacing-traditional-python-scripts-45feac26922f?source=rss----78073def27b8---4">Continue reading on Python in Plain English »</a></div>

In 2026, “writing a script” is no longer the primary goal — AI agents handle what scripts couldn’t.
Continue reading on Python in Plain English »

Agentic Automation — How AI Agents Are Replacing Traditional Python Scripts

Hey, there!
This is Kenn, Your Daily Advocate from CodeWithKenn!
Welcome to the Blog! Make yourself at home! 
In this article, we won't talk much. For a couple of days, I've been thinking about a way of making my content easier and shorter so that De...

Hey, there!
This is Kenn, Your Daily Advocate from CodeWithKenn!
Welcome to the Blog! Make yourself at home! 
In this article, we won't talk much. For a couple of days, I've been thinking about a way of making my content easier and shorter so that Developers won't have to spend so much time scrolling; I want this Blog a Place of Resources and Discovery, not a Boring Blog.
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1644857915236/HNJherNb5.jpeg" alt="save-your-time-2a7fb4.jpg" />
<h1 id="heading-what-are-python-lists">What are Python Lists?</h1>
Today, we're gonna get one of the first resources, Python Lists Easy Cheatsheet.
Python Lists are used to store an ordered collection of items, which might be of different types but usually they aren't.
<h1 id="heading-cheatsheet-content">Cheatsheet Content</h1>
<h2 id="heading-step-1">Step 1</h2>
<h3 id="heading-snippet-image">Snippet - Image</h3>
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1644858459662/CGjzyuSpc.png" alt="carbon (1).png" />
<h3 id="heading-snippet-code">Snippet - Code</h3>
<pre><code># Create an empty list using square brackets.
numbers = []
print(numbers) # Output: []

# Create an empty list using list().
numbers = list()
print(numbers) # Output: []

# Create a list of numbers.
numbers = [1, 2, 3]
print(numbers) # Output: [1, 2, 3]

# Create a list of numbers in a range.
numbers = list(range(1, 4))
print(numbers) # Output: [1, 2, 3]

# Create a list of tuples.
tuples_list = [(1, 2), (2, 4), (3, 6)]
print(tuples_list) # Output: [(1, 2), (2, 4), (3, 6)]
</code></pre><h2 id="heading-step-2">Step 2</h2>
<h3 id="heading-snippet-image">Snippet - Image</h3>
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1644858667007/4gPld_KaY.png" alt="carbon (2).png" />
<h3 id="heading-snippet-code">Snippet - Code</h3>
<pre><code># Create a list of lists.
list_of_lists = [[1, 2], [2, 4], [3, 6]]
print(list_of_lists) # Output: [[1, 2], [2, 4], [3, 6]]

# Create a list with items of different data types.
random_list = [1, "hey", [1, 2]]
print(random_list) # Output: [1, "hey", [1, 2]]

# Get length of list by using len() method.
numbers = [5, 8, 8]
print(len(numbers)) # Output: 3

# Access elements of a list by indexing.
str_list = ["hey", "there!", "how", "are", "you?"]
print(str_list[0]) # Output: "hey"
print(str_list[len(str_list) - 1]) # Output: "you?"
print(str_list[-1]) # Output: "you?"
</code></pre><h2 id="heading-step-3">Step 3</h2>
<h3 id="heading-snippet-image">Snippet - Image</h3>
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1644858847484/c0ND1W-Sd.png" alt="carbon (3).png" />
<h3 id="heading-snippet-code">Snippet - Code</h3>
<pre><code># Slicing a list.
str_list = ["hey", "there!", "how", "are", "you?"]
print(str_list[2:]) # Output: ["how", "are", "you?"]
print(str_list[:2]) # Output: ["hey", "there!"]
print(str_list[-3:]) # Output: ["how", "are", "you?"]
print(str_list[:-3]) # Output: ["hey", "there!"]
print(str_list[1:4]) # Output: ["there!", "how", "are"]
# Get a copy of list by slicing.
print(str_list[:]) # Output: ["hey", "there!", "how", "are", "you?"]
</code></pre><h2 id="heading-step-4">Step 4</h2>
<h3 id="heading-snippet-image">Snippet - Image</h3>
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1644859067103/nPyMnPAEf.png" alt="carbon (4).png" />
<h3 id="heading-snippet-code">Snippet - Code</h3>
<pre><code>
# Append to a list.
numbers = [1, 2]
print(numbers) # Output: [1, 2]
numbers.append(3)
print(numbers) # Output: [1, 2, 3]

# Concatenate lists.
numbers = [1, 2]
strings = ["Hey", "there"]
print(numbers + strings) # Output: [1, 2, "Hey", "there"]
</code></pre><h2 id="heading-step-5">Step 5</h2>
<h3 id="heading-snippet-image">Snippet - Image</h3>
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1644859171328/c4_s-MMOg.png" alt="carbon (5).png" />
<h3 id="heading-snippet-code">Snippet - Code</h3>
<pre><code># Mutate a list, that is, change its contents.
numbers = [1, 2, 3]
numbers[0] = 100
print(numbers) # Output: [100, 2, 3]
numbers[0:2] = [300, 400]
print(numbers) # Output: [300, 400, 3]
numbers[1:3] = []
print(numbers) # Output: [300]
numbers[:] = []
print(numbers) # Output: []

# Insert item to a list.
greeting = ["how", "you?"]
greeting.insert(1, "are")
print(greeting) # Output: ["how", "are", "you?"]
</code></pre><h2 id="heading-step-6">Step 6</h2>
<h3 id="heading-snippet-image">Snippet - Image</h3>
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1644859543800/asWaM8DPR.png" alt="carbon-lists-2.png" />
<h3 id="heading-snippet-code">Snippet - Code</h3>
<pre><code># Our list of lists.
matrix_1 = [[1,1,1], [2,2,2], [3,3,3]]
matrix_2 = [[4,4,4], [5,5,5], [6,6,6]]

# Matrix addition with for loop.
# Assuming that the matrices are of the same dimensions
matrix_sum = []
for row in range(len(matrix_1)):
 matrix_sum.append([])
 for column in range(len(matrix[0])):
 matrix_sum[row].append(matrix_1[row][column] + matrix_2[row][column])
</code></pre><h2 id="heading-step-7">Step 7</h2>
<h3 id="heading-snippet-image">Snippet - Image</h3>
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1644859582127/JjYy8T4TYT.png" alt="carbon-lists-2-1.png" />
<h3 id="heading-snippet-code">Snippet - Code</h3>
<pre><code>
# Our list of lists.
matrix_1 = [[1,1,1], [2,2,2], [3,3,3]]
matrix_2 = [[4,4,4], [5,5,5], [6,6,6]]


# Rewrite using list comprehension only for inner lists.
matrix_sum = []
for row in range(len(matrix_1)):
 matrix_sum.append([matrix_1[row][column] + matrix_2[row][column] for column in range(len(matrix[0]))])

# Rewrite using nested list comprehension.
matrix_sum = [[matrix_1[row][column] + matrix_2[row][column] for column in range(len(matrix[0]))]
 for row in range(len(matrix_1))]

print(matrix_sum) # Output: [[5, 5, 5], [7, 7, 7], [9, 9, 9]]
</code></pre><h2 id="heading-credit-cheat-sheet">Credit - Cheat Sheet</h2>
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1644859646012/41yYXVXuE.png" alt="carbon.png" />
⭐ All the Credit goes to <a target="_blank" href="https://gist.github.com/ShyamaSankar">@ShyamaSankar</a>
<h1 id="heading-before-leaving">🌎 Before Leaving</h1>
<ul>
<li>⚡ <a target="_blank" href="https://codewithkenn.hashnode.dev/discover-amazing-python-resources-for-free">Discover Amazing Python Resources FOR FREE!</a>
</li>
<li>⚡ <a target="_blank" href="https://codewithkenn.hashnode.dev/you-need-to-learn-python-in-2022-business-perspective">YOU NEED to Learn Python in 2022! (Business Perspective)</a>
</li>
</ul>
<ul>
<li><a target="_blank" href="https://twitter.com/CodeWithKenn">Twitter</a> </li>
<li><a target="_blank" href="https://github.com/KennStack01">Github</a> </li>
<li><a target="_blank" href="https://www.linkedin.com/in/kennkibadi/">LinkedIn</a> </li>
<li><a target="_blank" href="https://www.instagram.com/codewithkenn_/">Instagram</a> </li>
<li><a target="_blank" href="https://www.facebook.com/Code-With-Kenn-104376625415201">Facebook</a></li>
<li><a target="_blank" href="https://www.bio.link/IamKenn">See My Bio Link</a></li>
</ul>
Want to start blogging? 🔥<a target="_blank" href="https://hashnode.com/@CodeWithKenn/joinme">Join NOW!</a>

Hey, there!
This is Kenn, Your Daily Advocate from CodeWithKenn!
Welcome to the Blog! Make yourself at home! 
In this article, we won't talk much. For a couple of days, I've been thinking about a way of making my content easier and shorter so that Developers won't have to spend so much time scrolling; I want this Blog a Place of Resources and Discovery, not a Boring Blog.

What are Python Lists?
Today, we're gonna get one of the first resources, Python Lists Easy Cheatsheet.
Python Lists are used to store an ordered collection of items, which might be of different types but usually they aren't.
Cheatsheet Content
Step 1
Snippet - Image

Snippet - Code
# Create an empty list using square brackets.
numbers = []
print(numbers) # Output: []

# Create an empty list using list().
numbers = list()
print(numbers) # Output: []

# Create a list of numbers.
numbers = [1, 2, 3]
print(numbers) # Output: [1, 2, 3]

# Create a list of numbers in a range.
numbers = list(range(1, 4))
print(numbers) # Output: [1, 2, 3]

# Create a list of tuples.
tuples_list = [(1, 2), (2, 4), (3, 6)]
print(tuples_list) # Output: [(1, 2), (2, 4), (3, 6)]

Step 2
Snippet - Image

Snippet - Code
# Create a list of lists.
list_of_lists = [[1, 2], [2, 4], [3, 6]]
print(list_of_lists) # Output: [[1, 2], [2, 4], [3, 6]]

# Create a list with items of different data types.
random_list = [1, "hey", [1, 2]]
print(random_list) # Output: [1, "hey", [1, 2]]

# Get length of list by using len() method.
numbers = [5, 8, 8]
print(len(numbers)) # Output: 3

# Access elements of a list by indexing.
str_list = ["hey", "there!", "how", "are", "you?"]
print(str_list[0]) # Output: "hey"
print(str_list[len(str_list) - 1]) # Output: "you?"
print(str_list[-1]) # Output: "you?"

Step 3
Snippet - Image

Snippet - Code
# Slicing a list.
str_list = ["hey", "there!", "how", "are", "you?"]
print(str_list[2:]) # Output: ["how", "are", "you?"]
print(str_list[:2]) # Output: ["hey", "there!"]
print(str_list[-3:]) # Output: ["how", "are", "you?"]
print(str_list[:-3]) # Output: ["hey", "there!"]
print(str_list[1:4]) # Output: ["there!", "how", "are"]
# Get a copy of list by slicing.
print(str_list[:]) # Output: ["hey", "there!", "how", "are", "you?"]

Step 4
Snippet - Image

Snippet - Code

# Append to a list.
numbers = [1, 2]
print(numbers) # Output: [1, 2]
numbers.append(3)
print(numbers) # Output: [1, 2, 3]

# Concatenate lists.
numbers = [1, 2]
strings = ["Hey", "there"]
print(numbers + strings) # Output: [1, 2, "Hey", "there"]

Step 5
Snippet - Image

Snippet - Code
# Mutate a list, that is, change its contents.
numbers = [1, 2, 3]
numbers[0] = 100
print(numbers) # Output: [100, 2, 3]
numbers[0:2] = [300, 400]
print(numbers) # Output: [300, 400, 3]
numbers[1:3] = []
print(numbers) # Output: [300]
numbers[:] = []
print(numbers) # Output: []

# Insert item to a list.
greeting = ["how", "you?"]
greeting.insert(1, "are")
print(greeting) # Output: ["how", "are", "you?"]

Step 6
Snippet - Image

Snippet - Code
# Our list of lists.
matrix_1 = [[1,1,1], [2,2,2], [3,3,3]]
matrix_2 = [[4,4,4], [5,5,5], [6,6,6]]

# Matrix addition with for loop.
# Assuming that the matrices are of the same dimensions
matrix_sum = []
for row in range(len(matrix_1)):
  matrix_sum.append([])
  for column in range(len(matrix[0])):
      matrix_sum[row].append(matrix_1[row][column] + matrix_2[row][column])

Step 7
Snippet - Image

Snippet - Code

# Our list of lists.
matrix_1 = [[1,1,1], [2,2,2], [3,3,3]]
matrix_2 = [[4,4,4], [5,5,5], [6,6,6]]


# Rewrite using list comprehension only for inner lists.
matrix_sum = []
for row in range(len(matrix_1)):
  matrix_sum.append([matrix_1[row][column] + matrix_2[row][column] for column in range(len(matrix[0]))])

# Rewrite using nested list comprehension.
matrix_sum = [[matrix_1[row][column] + matrix_2[row][column] for column in range(len(matrix[0]))]
              for row in range(len(matrix_1))]

print(matrix_sum) # Output: [[5, 5, 5], [7, 7, 7], [9, 9, 9]]

Credit - Cheat Sheet

⭐ All the Credit goes to @ShyamaSankar
🌎 Before Leaving
⚡ Discover Amazing Python Resources FOR FREE!
⚡ YOU NEED to Learn Python in 2022! (Business Perspective)
Twitter 
Github 
LinkedIn 
Instagram 
Facebook
See My Bio Link
Want to start blogging?  🔥Join NOW!

Python Lists Cheat Sheet - Developer Resources

What if I told you You can learn Python in ONLY three weeks? 

Will you trust me? haha 😅
Hey, there! 
This is Kenn, Your Daily Advocate, and Friend from CodeWithKenn!
In this Article, You gonna get one of the easiest Python Learning Roadmap and Reso...

What if I told you You can learn Python in ONLY three weeks? 
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1642666266619/IWrUSxbAn.jpeg" alt="a4a7c2e1507c539f41229ca8200d3a44.jpg" />
Will you trust me? haha 😅
Hey, there! 
This is Kenn, Your Daily Advocate, and Friend from CodeWithKenn!
In this Article, You gonna get one of the easiest Python Learning Roadmap and Resources for Free. Make sure to follow up for more articles coming. That's it! Let's get down to Business!
Don't worry, You won't get bored with CodeWithKenn! Easy, Free, Short, but Use articles for Your Journey. Let's Gooooooo!
<h1 id="heading-brief-description-of-python">Brief Description of Python</h1>
If you just come from my article on <a target="_blank" href="https://codewithkenn.hashnode.dev/you-need-to-learn-python-in-2022-business-perspective">Learning Python in 2022! (Business Perspective)</a>, You've already known much on the subject.
In a nutshell, Python is one of the easiest programming languages to get started with. For this reason, the community keeps growing every month, with a market value of over $10 Billion.
So, why not get the Resources you really need for the Journey???
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1642666622218/c1IFjXUpB.jpeg" alt="4752b206e5dc2c3713d57f536d527d06.jpg" />
<h1 id="heading-get-what-you-really-need">Get what You really need</h1>
As they are plenty of them, here are the two main resources you can use as references to go and check according to what you need:
<h2 id="heading-refresh-the-memory-with-the-fundamentals">Refresh the memory with the Fundamentals</h2>
Without the fundamentals, there is no way to bring innovation, meaning there is a necessity of getting a foundation. Here are the main resources:
<h4 id="heading-videos-resources">Videos Resources</h4>
⚡ <a target="_blank" href="https://www.youtube.com/watch?v=rfscVS0vtbw">FreeCodeCamp Beginners Course</a> 
⚡ <a target="_blank" href="https://www.youtube.com/playlist?list=PL-osiE80TeTskrapNbzXhwoFUiLCjGgY7">Corey Schafer Beginners Python Videos Series</a> 
⚡ <a target="_blank" href="https://www.youtube.com/watch?v=8ext9G7xspg">Start Building Projects</a> 
<h4 id="heading-community-resources">Community Resources</h4>
⚡ <a target="_blank" href="https://forum.freecodecamp.org/c/python/424">FreeCodeCamp Python Community</a> 
⚡ <a target="_blank" href="https://www.reddit.com/r/Python/">Python Sub-reddit to ask for help</a> 
<h4 id="heading-docs-resources">Docs Resources</h4>
⚡ <a target="_blank" href="https://docs.python.org/3/tutorial/index.html">Python Official Docs</a> 
⚡ <a target="_blank" href="https://devdocs.io/python~3.10-tutorial/">DevDocs Python</a> 
<h2 id="heading-python-for-web-development">Python for Web Development</h2>
Here you have all You need for your Backend Journey. Don't overthink, use this and start building stuff:
<h3 id="heading-interested-in-flask-app-development">Interested in Flask App Development?</h3>
<h4 id="heading-videos-resources">Videos Resources</h4>
⚡ <a target="_blank" href="https://www.youtube.com/playlist?list=PL-osiE80TeTs4UjLw5MM6OjgkjFeUxCYH">Corey Schafer Beginners Flask Videos Series</a> 
⚡ <a target="_blank" href="https://www.youtube.com/watch?v=yKHJsLUENl0">Build a CRUD APP with Python Engineer</a> 
<h4 id="heading-community-resources">Community Resources</h4>
Coding, but Stuck with some bugs? Here is the Solution, Go and ask the Community:
⚡ <a target="_blank" href="https://www.reddit.com/r/flask/">Flask Subreddit Community</a>
⚡ <a target="_blank" href="https://www.reddit.com/r/Python/">Python Subreddit Community</a>
<h4 id="heading-docs-resources">Docs Resources</h4>
⚡ <a target="_blank" href="https://github.com/humiaozuzu/awesome-flask">Awesome Flask Links</a> 
⚡ <a target="_blank" href="https://github.com/realpython/discover-flask">Flask for your Project</a> 
⚡ <a target="_blank" href="https://flask.palletsprojects.com/en/latest/">Flask Latest Docs</a> 
<h3 id="heading-django-app-development">Django App Development</h3>
<h4 id="heading-videos-resources">Videos Resources</h4>
⚡ <a target="_blank" href="https://www.youtube.com/playlist?list=PL-osiE80TeTtoQCKZ03TU5fNfx2UY6U4p">Corey Schafer Beginners Django Videos Series</a> 
⚡ <a target="_blank" href="https://www.youtube.com/watch?v=F5mRW0jo-U4&amp;t=64s">FreeCodeCamp Djajngo for Beginners</a> 
<h4 id="heading-community-resources">Community Resources</h4>
Coding, but Stuck with some bugs? Here is the Solution, Go and ask the Community:
⚡ <a target="_blank" href="https://www.reddit.com/r/django/">Django Subreddit Community</a>
⚡ <a target="_blank" href="https://www.djangoproject.com/community/blogs/">Django Community</a>
<h4 id="heading-docs-resources">Docs Resources</h4>
⚡ <a target="_blank" href="https://docs.djangoproject.com/en/4.0/intro/">Django Official Docs</a> 
⚡ <a target="_blank" href="https://django.readthedocs.io/en/stable/contents.html">Django ReadTheDocs.io</a> 
🔥 This article is being updated, I'm adding Fast API Resources and AI/ML Resources on the Blog! Stay tuned! 
Stay tuned! More articles are coming out! Feel free to comment and share the articles to support me 🤙
<h1 id="heading-lets-connect">🌎 Let's connect</h1>
<ul>
<li><a target="_blank" href="https://twitter.com/CodeWithKenn">Twitter</a> </li>
<li><a target="_blank" href="https://github.com/KennStack01">Github</a> </li>
<li><a target="_blank" href="https://www.linkedin.com/in/kennkibadi/">LinkedIn</a> </li>
<li><a target="_blank" href="https://www.instagram.com/codewithkenn_/">Instagram</a> </li>
<li><a target="_blank" href="https://www.facebook.com/Code-With-Kenn-104376625415201">Facebook</a></li>
</ul>
Want to start blogging? 🔥<a target="_blank" href="https://hashnode.com/@CodeWithKenn/joinme">Join NOW!</a>

What if I told you You can learn Python in ONLY three weeks? 

Will you trust me? haha 😅
Hey, there! 
This is Kenn, Your Daily Advocate, and Friend from CodeWithKenn!
In this Article, You gonna get one of the easiest Python Learning Roadmap and Resources for Free. Make sure to follow up for more articles coming. That's it! Let's get down to Business!
Don't worry, You won't get bored with CodeWithKenn! Easy, Free, Short, but Use articles for Your Journey. Let's Gooooooo!
Brief Description of Python
If you just come from my article on Learning Python in 2022! (Business Perspective), You've already known much on the subject.
In a nutshell, Python is one of the easiest programming languages to get started with. For this reason, the community keeps growing every month, with a market value of over $10 Billion.
So, why not get the Resources you really need for the Journey???

Get what You really need
As they are plenty of them, here are the two main resources you can use as references to go and check according to what you need:
Refresh the memory with the Fundamentals
Without the fundamentals, there is no way to bring innovation, meaning there is a necessity of getting a foundation. Here are the main resources:
Videos Resources
⚡ FreeCodeCamp Beginners Course 
⚡ Corey Schafer Beginners Python Videos Series 
⚡ Start Building Projects 
Community Resources
⚡ FreeCodeCamp Python Community 
⚡ Python Sub-reddit to ask for help 
Docs Resources
⚡ Python Official Docs 
⚡ DevDocs Python 
Python for Web Development
Here you have all You need for your Backend Journey. Don't overthink, use this and start building stuff:
Interested in Flask App Development?
Videos Resources
⚡ Corey Schafer Beginners Flask Videos Series 
⚡ Build a CRUD APP with Python Engineer 
Community Resources
Coding, but Stuck with some bugs? Here is the Solution, Go and ask the Community:
⚡ Flask Subreddit Community
⚡ Python Subreddit Community
Docs Resources
⚡ Awesome Flask Links 
⚡ Flask for your Project 
⚡ Flask Latest Docs 
Django App Development
Videos Resources
⚡ Corey Schafer Beginners Django Videos Series 
⚡ FreeCodeCamp Djajngo for Beginners 
Community Resources
Coding, but Stuck with some bugs? Here is the Solution, Go and ask the Community:
⚡ Django Subreddit Community
⚡ Django Community
Docs Resources
⚡ Django Official Docs 
⚡ Django ReadTheDocs.io 
🔥 This article is being updated, I'm adding Fast API Resources and AI/ML Resources on the Blog! Stay tuned! 
Stay tuned! More articles are coming out! Feel free to comment and share the articles to support me 🤙
🌎 Let's connect
Twitter 
Github 
LinkedIn 
Instagram 
Facebook
Want to start blogging?  🔥Join NOW!

Discover Amazing Python Resources FOR FREE!

Python is one of the easiest programming languages to get started with. For this reason, the community keeps growing every month.
Because of its simplicity (of the Syntax), Python has become the most popular language.
Besides, Python helps Engineers ...

Python is one of the easiest programming languages to get started with. For this reason, the community keeps growing every month.
Because of its simplicity (of the Syntax), Python has become the most popular language.
Besides, Python helps Engineers build great products in many industries and professional domains.
The Programmer's life gets easier when it comes to having a look at Python.
<h1 id="heading-what-about-the-python-business-market-size">What about the Python Business Market size?</h1>
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1642662070916/ha3UIJDHx.jpeg" alt="market-size-market-share.jpg" />
Python is an open-source language, which means that anyone can use it. However, if you look to an industry such as data analytics, in which Python can be used, the market value is over $10 billion.
Shared by <a target="_blank" href="https://www.futurelearn.com/info/insights/python-insights">futurelearn.com</a> 
<h1 id="heading-here-are-the-top-10-most-common-uses-of-python-by-industry">Here are the top 10 most common uses of Python by industry</h1>
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1642662192938/xYTgi2Ycs.png" alt="pandemic-hero.png" />
▪ Insurance: Creating business insights with machine learning.
▪ Retail Banking: Flexible data transformation and manipulation.
▪ Aerospace: Meeting software system deadlines.
▪ Finance: Data mining to identify cross-sell opportunities.
▪ Business Services: API access to financial information
▪ Hardware: Automating network administration
▪ Healthcare: Predicting disease prognosis
▪ Consulting Services: Bespoke web development
▪ InfoTech: IT modernization
▪ Software: Adding extensibility to legacy applications
Shared by <a target="_blank" href="https://appdevelopermagazine.com/ten-industries-using-python-programming/">appdevelopermagazine.com</a> 
<h1 id="heading-here-are-5-world-class-software-companies-that-use-python">Here are 5 World-Class Software Companies That Use Python</h1>
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1642662333807/Y73E563_X.jpeg" alt="COTW-TechAcquisitions-Shareable-2.jpg" />
(Photo by Visual Capitalist)
Google, Meta (Facebook), Instagram, Netflix, Instagram.
There are plenty of companies searching for Python Developers around the globe.
<h1 id="heading-hence-i-created-this-blog-to-help-you-out">Hence, I created this Blog to help you out!</h1>
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1642662424540/nnlZkUO_y.jpeg" alt="Design-Blog-Page-Python.jpg" />
Hence, Learning Python helps you create Opportunities from jobs to Building Products people use.
I just got into Full-stack development using Python Software Development, React + Python as Tech Stack.
<h2 id="heading-make-sure-to-follow-codewithkenn-for-more-articles-about">Make sure to follow CodeWithKenn for more articles about:</h2>
➞ Learning Python in simple terms
➞ Building Side Projects and SaaS using React + Python
➞ Getting useful Resources and tips for your coding Journey
Stay tuned! More articles are coming out! Feel free to comment and share the articles to support me 🤙
<h1 id="heading-lets-connect">🌎 Let's connect</h1>
<ul>
<li><a target="_blank" href="https://twitter.com/CodeWithKenn">Twitter</a> </li>
<li><a target="_blank" href="https://github.com/KennStack01">Github</a> </li>
<li><a target="_blank" href="https://www.linkedin.com/in/kennkibadi/">LinkedIn</a> </li>
<li><a target="_blank" href="https://www.instagram.com/codewithkenn_/">Instagram</a> </li>
<li><a target="_blank" href="https://www.facebook.com/Code-With-Kenn-104376625415201">Facebook</a></li>
</ul>
Want to start blogging? 🔥<a target="_blank" href="https://hashnode.com/@CodeWithKenn/joinme">Join NOW!</a>

Python is one of the easiest programming languages to get started with. For this reason, the community keeps growing every month.
Because of its simplicity (of the Syntax), Python has become the most popular language.
Besides, Python helps Engineers build great products in many industries and professional domains.
The Programmer's life gets easier when it comes to having a look at Python.
What about the Python Business Market size?

Python is an open-source language, which means that anyone can use it. However, if you look to an industry such as data analytics, in which Python can be used, the market value is over $10 billion.
Shared by futurelearn.com 
Here are the top 10 most common uses of Python by industry

▪ Insurance: Creating business insights with machine learning.
▪ Retail Banking: Flexible data transformation and manipulation.
▪ Aerospace: Meeting software system deadlines.
▪ Finance: Data mining to identify cross-sell opportunities.
▪ Business Services: API access to financial information
▪ Hardware: Automating network administration
▪ Healthcare: Predicting disease prognosis
▪ Consulting Services: Bespoke web development
▪ InfoTech: IT modernization
▪ Software: Adding extensibility to legacy applications
Shared by appdevelopermagazine.com 
Here are 5 World-Class Software Companies That Use Python

(Photo by Visual Capitalist)
Google, Meta (Facebook), Instagram, Netflix, Instagram.
There are plenty of companies searching for Python Developers around the globe.
Hence, I created this Blog to help you out!

Hence, Learning Python helps you create Opportunities from jobs to Building Products people use.
I just got into Full-stack development using Python Software Development, React + Python as Tech Stack.
Make sure to follow CodeWithKenn for more articles about:
➞ Learning Python in simple terms
➞ Building Side Projects and SaaS using React + Python
➞ Getting useful Resources and tips for your coding Journey
Stay tuned! More articles are coming out! Feel free to comment and share the articles to support me 🤙
🌎 Let's connect
Twitter 
Github 
LinkedIn 
Instagram 
Facebook
Want to start blogging?  🔥Join NOW!

YOU NEED to Learn Python in 2022! (Business Perspective)

Python is one of the most popular coding languages available and is great as a starting point in your learning journey. It can be used for a range of things, such as web and internet development, software development application, network programming,...

Python is one of the most popular coding languages available and is great as a starting point in your learning journey. It can be used for a range of things, such as web and internet development, software development application, network programming, and 3D graphics... (<a target="_blank" href="https://hethelinnovation.com/">hethelinnovation.com</a>)
While learning, we end up getting stuck in small bugs that can take more time to solve, whereas we could have solved it quickly if we could only remember the basics. 
So, this is a series of Basics Python tips to remember whenever we work with numbers and strings. You can also use this article as a reference somehow while coding...
Enjoy!
<h1 id="heading-python-numbers">Python Numbers</h1>
There are three numeric types we use in Python: 
<ul>
<li>int: or integer, is a whole number without decimals.</li>
</ul>
<pre><code>x = 1245
y = -789

print(type(x))
print(type(y))
</code></pre><ul>
<li>float: or Floating, a number with one or more decimals.</li>
</ul>
<pre><code>x = 12.45
y = 42E5

print(type(x))
print(type(y))
</code></pre><ul>
<li>complex: as studied in high school, is a number written with a "j" as the imaginary part.</li>
</ul>
<pre><code>x = 3 + 2j
y = -15j

print(type(x))
print(type(y))
</code></pre><h2 id="heading-numbers-types-conversion">Numbers Types Conversion</h2>
To convert numbers from one type (from the three) to another, we used methods.
In Python, a method is a function that is available for a given object because of the object's type.
So, You can convert from one type to another with the int(), float(), and complex() methods:
<pre><code>
x = 4 # int number
y = 0.8 # float number
z = 4j # complex number

# from int to float:
a = float(x)

# from float to int:
b = int(y)

# from int to complex:
c = complex(x)

print(a)
print(b)
print(c)

print(type(a))
print(type(b))
print(type(c))
</code></pre><h2 id="heading-lets-build-a-simple-calculator">Let's Build a Simple Calculator</h2>
<pre><code># Python Simple calculator

# Addition Function
def add(x, y):
 return x + y

# Substraction Function
def subtract(x, y):
 return x - y

# Multiplication Function
def multiply(x, y):
 return x * y

# Division Function
def divide(x, y):
 return x / y


def calculate(first_number, second_number, operator):
 if operator == "+":
 answer = add(first_number, second_number)
 return answer

 elif operator == "-":
 answer = subtract(first_number, second_number)
 return answer

 elif operator == "/":
 answer = divide(first_number, second_number)
 return answer
 elif operator == "*" or operator == "x":
 answer = subtract(first_number, second_number)
 return answer
 else:
 return "Invalid"


num1 = int(input("Enter the first number: "))
num2 = int(input("Enter the second number: "))
operator = input("Enter the Operator: ")

print(f"{num1} {operator} {num2} = {calculate(num1, num2, operator)}")
</code></pre><h1 id="heading-python-strings">Python Strings</h1>
Strings in python are surrounded by: 
<ul>
<li>Single quotation marks 
<pre><code>'bonjour'
</code></pre></li>
<li>Double quotation marks
<pre><code>"bonjour"
</code></pre></li>
</ul>
<h2 id="heading-use-a-variable-to-save-the-string">Use a Variable to save the String</h2>
The assignment sign is made of = (equal sign).
<pre><code>name = "Kennedy"
role = "Software Engineer"
print(name)
print(role)
</code></pre><h2 id="heading-we-can-also-use-multiline-strings">We can also use Multiline Strings</h2>
<pre><code>poem = """Your face is the grave of your nose
your face is the grave of your ears
your face is the grave of your face
once again your face overflows uncontrollably."""

print(poem)
</code></pre><h2 id="heading-accessing-a-character-of-the-string">Accessing a Character of the String</h2>
We can think of strings like an array made of characters. For example, the word "Elon Musk" is made of "E", "l", "o", "n", " ", "M", "u", "s" and "k" (Notice the space is also counted as a character).
The first character is counted at index 0 (not 1). 
<pre><code>greeting = "Hello, World!"
print(greeting[1]) #prints "e"
</code></pre><h2 id="heading-working-with-strings">Working with Strings</h2>
<h3 id="heading-checking-the-string-length">Checking the String Length</h3>
use the <code>len()</code> function
<pre><code>name = "Kennedy"
length_of_name = len(name)

print(length_of_name)
</code></pre><h3 id="heading-checking-an-existing-string-or-character-in-a-string">Checking an existing string or character in a string</h3>
Use the <code>in</code> keyword.
<pre><code>sentence = "Kennedy is a python programmer"
if "python" in a sentence:
 print("Yes, 'python' is present.")
</code></pre><h3 id="heading-most-common-string-methods">Most Common String Methods</h3>
<ul>
<li><code>capitalize()</code> method converts the first character of a string to an uppercase letter and lowercases all other characters.</li>
</ul>
<pre><code>name = "python"
print(name.capitalize())
</code></pre><ul>
<li><code>casefold()</code> method Converts string into lower case.</li>
</ul>
<pre><code>name = "PYTHON"
print(name.casefold())
</code></pre><ul>
<li><code>upper()</code> method converts all the characters into Uppercase.</li>
</ul>
<pre><code>name = "python"
print(name.upper())
</code></pre><ul>
<li><code>lower()</code> method converts all the characters into Lowercase.</li>
</ul>
<pre><code>name = "PYTHON"
print(name.lower())
</code></pre><ul>
<li><code>len()</code> method used to count the total number of characters in a string.</li>
</ul>
<pre><code>name = "python"
print( len(name) )
</code></pre><ul>
<li><code>find()</code> searches the string for a specified value and returns the position of where it was found.</li>
</ul>
<pre><code>sentence = "python is great"
print( sentence.find('great') )
</code></pre><ul>
<li><code>replace()</code> is used to replace a string with another.</li>
</ul>
<pre><code>sentence = "python is great"
new_sentence = sentence.replace('great', 'awesome') 
print(new_sentence)
</code></pre><ul>
<li><code>str()</code> is used for string conversion</li>
</ul>
<pre><code>ten = str(10)
print(ten) #converts 10 to '10'
</code></pre><ul>
<li>You can find more in the <a target="_blank" href="https://docs.python.org/2.5/lib/string-methods.html">Python.org Official Docs</a> </li>
</ul>
<h3 id="heading-lets-work-on-a-small-project">Let's work on a Small Project</h3>
<h4 id="heading-arrange-string-characters-such-that-lowercase-letters-should-come-first">- Arrange string characters such that lowercase letters should come first</h4>
Initial Code
<pre><code>noun = PrOgRamMinG

# Expected Output : rgaminPORMG
</code></pre>Solution
<pre><code>noun = "PrOgRamMinG"

# Let's create a Function to handle it

def arrange_string(my_string):
 print('Original String:', my_string)
 lower = []
 upper = []

 #Let's iterate and convert
 for char in my_string:
 if char.islower():
 # add lowercase characters to lower list
 lower.append(char)
 else:
 # add uppercase characters to lower list
 upper.append(char)

 # Join both list
 sorted_str = ''.join(lower + upper)
 print('Result:', sorted_str)

# Now, let's call execute the function we just created! 
arrange_string(noun)

# Output: 
# Original String: PrOgRamMinG
# Result: rgaminPORMG
</code></pre>Resources for more Details:
<ul>
<li><a target="_blank" href="https://www.w3schools.com/python/">www.w3schools.com</a> </li>
<li><a target="_blank" href="https://www.programiz.com/python-programming">www.programiz.com</a> </li>
<li><a target="_blank" href="https://pynative.com/python-string-exercise/#h-exercise-4-arrange-string-characters-such-that-lowercase-letters-should-come-first">www.pynative.com/</a> </li>
</ul>
👉 <a target="_blank" href="https://codewithkenn.hashnode.dev/">DISCOVER MORE useful ARTICLES</a> 
Thanks for reading this article, many others are coming very soon, Feel free to subscribe 🤙.
<h1 id="heading-lets-connect">🌎 Let's connect</h1>
<ul>
<li><a target="_blank" href="https://twitter.com/CodeWithKenn">Twitter</a> </li>
<li><a target="_blank" href="https://github.com/KennStack01">Github</a> </li>
<li><a target="_blank" href="https://www.linkedin.com/in/kennkibadi/">LinkedIn</a> </li>
<li><a target="_blank" href="https://www.instagram.com/codewithkenn_/">Instagram</a> </li>
</ul>
Want to start blogging? 🔥<a target="_blank" href="https://hashnode.com/@CodeWithKenn/joinme">Join NOW!</a>

Python is one of the most popular coding languages available and is great as a starting point in your learning journey. It can be used for a range of things, such as web and internet development, software development application, network programming, and 3D graphics... (hethelinnovation.com)
While learning, we end up getting stuck in small bugs that can take more time to solve, whereas we could have solved it quickly if we could only remember the basics. 
So, this is a series of Basics Python tips to remember whenever we work with numbers and strings. You can also use this article as a reference somehow while coding...
Enjoy!
Python Numbers
There are three numeric types we use in Python: 
int: or integer, is a whole number without decimals.
x = 1245
y = -789

print(type(x))
print(type(y))


float: or Floating, a number with one or more decimals.
x = 12.45
y = 42E5

print(type(x))
print(type(y))


complex: as studied in high school, is a number written with a "j" as the imaginary part.
x = 3 + 2j
y = -15j

print(type(x))
print(type(y))

Numbers Types Conversion
To convert numbers from one type (from the three) to another, we used methods.
In Python, a method is a function that is available for a given object because of the object's type.
So, You can convert from one type to another with the int(), float(), and complex() methods:

x = 4    # int number
y = 0.8  # float number
z = 4j   # complex number

# from int to float:
a = float(x)

# from float to int:
b = int(y)

# from int to complex:
c = complex(x)

print(a)
print(b)
print(c)

print(type(a))
print(type(b))
print(type(c))

Let's Build a Simple Calculator
# Python Simple calculator

# Addition Function
def add(x, y):
    return x + y

# Substraction Function
def subtract(x, y):
    return x - y

# Multiplication Function
def multiply(x, y):
    return x * y

# Division Function
def divide(x, y):
    return x / y


def calculate(first_number, second_number, operator):
    if operator == "+":
        answer = add(first_number, second_number)
        return answer

    elif operator == "-":
        answer = subtract(first_number, second_number)
        return answer

    elif operator == "/":
        answer = divide(first_number, second_number)
        return answer
    elif operator == "*" or operator == "x":
        answer = subtract(first_number, second_number)
        return answer
    else:
        return "Invalid"


num1 = int(input("Enter the first number: "))
num2 = int(input("Enter the second number: "))
operator = input("Enter the Operator: ")

print(f"{num1} {operator} {num2} = {calculate(num1, num2, operator)}")

Python Strings
Strings in python are surrounded by: 
Single quotation marks 
'bonjour'



Double quotation marks
"bonjour"



Use a Variable to save the String
The assignment sign is made of = (equal sign).
name = "Kennedy"
role = "Software Engineer"
print(name)
print(role)

We can also use Multiline Strings
poem = """Your face is the grave of your nose
your face is the grave of your ears
your face is the grave of your face
once again your face overflows uncontrollably."""

print(poem)

Accessing a Character of the String
We can think of strings like an array made of characters. For example, the word "Elon Musk" is made of "E", "l", "o", "n", " ", "M", "u", "s" and "k" (Notice the space is also counted as a character).
The first character is counted at index 0 (not 1). 
greeting = "Hello, World!"
print(greeting[1]) #prints "e"

Working with Strings
Checking the String Length
use the len() function
name = "Kennedy"
length_of_name = len(name)

print(length_of_name)

Checking an existing string or character in a string
Use the in keyword.
sentence = "Kennedy is a python programmer"
if "python" in a sentence:
  print("Yes, 'python' is present.")

Most Common String Methods
capitalize() method converts the first character of a string to an uppercase letter and lowercases all other characters.
name = "python"
print(name.capitalize())


casefold() method Converts string into lower case.
name = "PYTHON"
print(name.casefold())


upper() method converts all the characters into Uppercase.
name = "python"
print(name.upper())


lower() method converts all the characters into Lowercase.
name = "PYTHON"
print(name.lower())


len() method used to count the total number of characters in a string.
name = "python"
print( len(name) )


find() searches the string for a specified value and returns the position of where it was found.
sentence = "python is great"
print( sentence.find('great') )


replace() is used to replace a string with another.
sentence = "python is great"
new_sentence = sentence.replace('great', 'awesome') 
print(new_sentence)


str() is used for string conversion
ten = str(10)
print(ten) #converts 10 to '10'


You can find more in the Python.org Official Docs 
Let's work on a Small Project
- Arrange string characters such that lowercase letters should come first
Initial Code
noun = PrOgRamMinG

# Expected Output : rgaminPORMG

Solution
noun = "PrOgRamMinG"

# Let's create a Function to handle it

def arrange_string(my_string):
    print('Original String:', my_string)
    lower = []
    upper = []

    #Let's iterate and convert
    for char in my_string:
        if char.islower():
            # add lowercase characters to lower list
            lower.append(char)
        else:
            # add uppercase characters to lower list
            upper.append(char)

    # Join both list
    sorted_str = ''.join(lower + upper)
    print('Result:', sorted_str)

# Now, let's call execute the function we just created! 
arrange_string(noun)

# Output: 
# Original String: PrOgRamMinG
# Result: rgaminPORMG

Resources for more Details:
www.w3schools.com 
www.programiz.com 
www.pynative.com/ 
👉 DISCOVER MORE useful ARTICLES 
Thanks for reading this article, many others are coming very soon, Feel free to subscribe 🤙.
🌎 Let's connect
Twitter 
Github 
LinkedIn 
Instagram 
Want to start blogging?  🔥Join NOW!

Working with Numbers & Strings in Python

We live in a world full of resources and data. Being a Python Developer can't escape the trend. That being said, in this article you're going to get all the resources you need to get started for the Journey.
After learning Python basics, you'll find ...

We live in a world full of resources and data. Being a Python Developer can't escape the trend. That being said, in this article you're going to get all the resources you need to get started for the Journey.
After learning Python basics, you'll find yourself being overwhelmed with a lot of information, and this can be very hard to go on. It becomes easier to get trapped in the Tutorial Hell 🔥.
<h1 id="heading-brief-description-of-flask">Brief Description of Flask</h1>
Flask is a micro web framework written in Python. It is classified as a microframework because it does not require particular tools or libraries. 
In other words, The Flask Framework is a Python platform that provides a foundation for developing software applications using Python Programming language. 
If your goal is to use Python for Web Development apps, Flask is what you need (sort of) for the Journey. Flask helps you set the Backend of your Web App. It's equivalent to Django (Another Python Backend Framework), NodeJS, Ruby on Rails, and so on.
What is Special about Flask is its simplicity, there are not many things to get overwhelmed, You only get what you need, simply. 🤙
That being said, using your favorite front-end technology (React, Vue, Angular, Svelte, or whatever you prefer), before jumping to read on the resources, you really have to determine what you need, exactly. Because, as we just said at the beginning, there are tons of resources and information you don't need, but still it makes you overwhelmed... 
<h1 id="heading-check-these-resources">Check these Resources</h1>
As they are plenty of them, here are the two main resources you can use as references to go and check according to what you need:
<h3 id="heading-refresh-the-memory-with-the-fundamentals">Refresh the memory with the Fundamentals</h3>
Without the fundamentals, there is no way to bring innovation, meaning there is a necessity of getting a foundation. Here are the main resources:
 ⚡ <a target="_blank" href="https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-i-hello-world">The Flask Mega-Tutorial</a>
 ⚡ <a target="_blank" href="https://medium.com/javarevisited/5-best-python-flask-courses-for-beginners-2f262f8e23da">Five Best Python Flask Courses You need</a> 
<h3 id="heading-get-the-resources-you-need-to-build-real-world-projects">Get the Resources you need to Build Real-world Projects</h3>
Knowing the Fundamentals isn't sufficient for real-world projects, you need to know topics like plugins (to avoid reinventing the wheel). Get the resources here:
⚡ <a target="_blank" href="https://github.com/humiaozuzu/awesome-flask">Awesome Flask Links</a> 
⚡ <a target="_blank" href="https://github.com/realpython/discover-flask">Flask for your Project</a> 
<h3 id="heading-get-the-benefit-of-the-flask-community">Get the Benefit of the Flask Community</h3>
Coding, but Stuck with some bugs? Here is the Solution, Go and ask the Community:
 <a target="_blank" href="https://www.reddit.com/r/flask/">Flask Subreddit Community</a>
 <a target="_blank" href="https://www.reddit.com/r/Python/">Python Subreddit Community</a>
Thanks for reading this article, many others are coming very soon, Feel free to subscribe 🤙.
<h1 id="heading-lets-connect">🌎 Let's connect</h1>
<ul>
<li><a target="_blank" href="https://twitter.com/CodeWithKenn">Twitter</a> </li>
<li><a target="_blank" href="https://github.com/KennStack01">Github</a> </li>
<li><a target="_blank" href="https://www.linkedin.com/in/kennkibadi/">LinkedIn</a> </li>
<li><a target="_blank" href="https://www.instagram.com/codewithkenn_/">Instagram</a> </li>
</ul>
Want to start blogging? 🔥<a target="_blank" href="https://hashnode.com/@CodeWithKenn/joinme">Join NOW!</a>

We live in a world full of resources and data. Being a Python Developer can't escape the trend. That being said, in this article you're going to get all the resources you need to get started for the Journey.
After learning Python basics, you'll find yourself being overwhelmed with a lot of information, and this can be very hard to go on. It becomes easier to get trapped in the Tutorial Hell 🔥.
Brief Description of Flask
Flask is a micro web framework written in Python. It is classified as a microframework because it does not require particular tools or libraries. 
In other words, The Flask Framework is a Python platform that provides a foundation for developing software applications using Python Programming language. 
If your goal is to use Python for Web Development apps, Flask is what you need (sort of) for the Journey. Flask helps you set the Backend of your Web App. It's equivalent to Django (Another Python Backend Framework), NodeJS, Ruby on Rails, and so on.
What is Special about Flask is its simplicity, there are not many things to get overwhelmed, You only get what you need, simply. 🤙
That being said, using your favorite front-end technology (React, Vue, Angular, Svelte, or whatever you prefer), before jumping to read on the resources, you really have to determine what you need, exactly. Because, as we just said at the beginning, there are tons of resources and information you don't need, but still it makes you overwhelmed... 
Check these Resources
As they are plenty of them, here are the two main resources you can use as references to go and check according to what you need:
Refresh the memory with the Fundamentals
Without the fundamentals, there is no way to bring innovation, meaning there is a necessity of getting a foundation. Here are the main resources:
 ⚡ The Flask Mega-Tutorial
 ⚡ Five Best Python Flask Courses You need 
Get the Resources you need to Build Real-world Projects
Knowing the Fundamentals isn't sufficient for real-world projects, you need to know topics like plugins (to avoid reinventing the wheel). Get the resources here:
⚡ Awesome Flask Links 
⚡ Flask for your Project 
Get the Benefit of the Flask Community
Coding, but Stuck with some bugs? Here is the Solution, Go and ask the Community:
 Flask Subreddit Community
 Python Subreddit Community
Thanks for reading this article, many others are coming very soon, Feel free to subscribe 🤙.
🌎 Let's connect
Twitter 
Github 
LinkedIn 
Instagram 
Want to start blogging?  🔥Join NOW!

Amazing Flask Resources FOR FREE!

Stop letting bot traffic evict your customers' sessions. A simple Redis configuration switch from LRU to LFU solved our crawler problem, with a Django configuration example.

<div class="block-html">
 <iframe allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen="" frameborder="0" height="400" referrerpolicy="strict-origin-when-cross-origin" src="https://www.youtube.com/embed/lSNdp7qpqVo?si=dtY_d8ah_hTV6LM8" title="YouTube video player" width="750">
 </iframe>
</div>
<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 
 We've seen a dramatic increase in malicious AI/web crawlers with our clients.
 
 
 These crawlers aren't honoring things like
 <code>
 robots.txt
 </code>
 or headers to indicate a page shouldn't be 
crawled (think links to facets like those in search results) that cause crawlers to get lost in a web of mostly 
useless or duplicated content.
 
 
 On top of that annoyance they're making tens if not
 
 hundreds of requests per second
 
 for hours at a time.
 
 
 These sites are reasonably fast and in many cases are behind awesome cache systems like Cloudflare, so 
why is it impacting our clients negatively?
 
 
 
 User sessions!
 
 These bots are creating thousands of new sessions that push the real human user's sessions out of the cache.
 
 
 We're using
 <a href="https://docs.djangoproject.com/en/dev/topics/http/sessions/">
 Django's Sessions
 </a>
 , but this 
technique works for any browser session token where you want to retain the sessions of real human 
users for as long as possible and expire out these robotic sessions.
 
 <h2>
 Let's talk about sessions
 </h2>
 
 Our default Django site template defaults to using the
 <code>
 cached_db
 </code>
 backend. This tries to get the session from 
Django's default cache (redis in most cases), if it's not found it is retrieved from the database and stored in the 
cache for the next request.
 
 
 Because of these asshole crawlers we've had to rethink this default and instead only keep the session in a cache 
for some of our customers.
 
 
 On most sites it doesn't really matter if the session data is retained. The worst case being the user is forced to log in again.
 
 
 With other sites, the user's session is absolutely crucial to the business. A simple example is we have e-commerce clients 
who want to provide a nice experience to their users. They want a random Internet user to be able to add items to their 
cart over time and only make them login to actually checkout.
 
 
 You need a session for that and you need that session to persist in cache for a reasonable amount of time.
 
 <h2>
 Default Redis
 </h2>
 
 By default Redis will use as much memory as possible and not expire keys that have a TTL set, but when it can no longer store keys in 
RAM it will fail to allow you to create a new session. This often causes whatever session system you're using to raise an exception 
and your site is broken.
 
 
 LRU (Least Recently Used) cache expiration is probably the most commonly known way to expire data. It's
 <a href="https://docs.python.org/3/library/functools.html#functools.lru_cache">
 built into Python
 </a>
 and it is the default for systems like memcached. You could configure or customize your session 
handling to use an LRU mechanism to keep RAM usage under control, but then you still have a problem.
 
 
 When you get a half million bogus AI requests in an hour and you only have RAM to hold say half of that, your real human 
customer who painstakingly curated the perfect cart of high dollar premium widgets but had the audacity to take a break to make 
dinner returns to their browser to find an empty cart and a fresh session.
 
 
 And unless you have a monopoly on widgets, they're likely to take their business elsewhere. I mean, if you can't keep my cart for an hour
why would I trust you with more complicated things like widget manufacturing and shipping logistics!?!?
 
 <h2>
 LFU to the Rescue
 </h2>
 
 I didn't know this until recently but Redis can be configured in a "volatile" mode which causes it to remove items even if the 
keys have TTLs on them. Redis provides
 <a href="https://redis.io/docs/latest/develop/reference/eviction/#eviction-policies">
 several options here
 </a>
 one of which is
 <code>
 volatile-lru
 </code>
 which is what most people would probably choose.
 
 
 However, what we really need here is LFU (Least Frequently Used)! Let me explain.
 
 
 Each of these bots generates a new session and uses it exactly once. This means that any regular human user who 
browses around a couple of pages will see their session remain for far longer. Frequent users, such as the website staff, 
likely would never have their session expire if not for the TTL.
 
 
 To illustrate this a bit I used an LLM to help me quickly generate a
 <a href="https://github.com/revsys/lfu-redis-example">
 demo of LRU vs LFU expiration in Redis
 </a>
 .
 
 
 The demo shows how we can keep shoving single use session keys into a Redis configured for LFU and retain our sessions which are 
used as little as a second time.
 
 <h2>
 Configuring Redis for LFU
 </h2>
 
 To adjust Redis for LFU you need to adjust your
 <code>
 redis.conf
 </code>
 with these three parameters:
 
 <div class="codehilite">
 <pre><code>maxmemory 9mb
maxmemory-policy volatile-lfu 
lazyfree-lazy-eviction yes
</code></pre>
 </div>
 
 This constrains Redis to use at most 9MBs of memory. Obviously you're going to want more session storage than that, so 
adjust it to as much RAM as makes sense for your situation. The next two turn on LFU even if the key has a TTL set (volatile) 
and tells Redis to expire keys asynchronously rather than blocking during the expiration.
 
 
 To do this in Docker Compose is fairly easy as well. You just need to write your own config and load it like this:
 
 <div class="codehilite">
 <pre><code>redis-lfu:
 image: "redis:8.2-alpine"
 command: redis-server /usr/local/redis/redis.conf
 ports:
 - "6001:6379"
 init: true
 stop_signal: SIGKILL
 volumes:
 - ./docker/redis-lfu.conf:/usr/local/redis/redis.conf:ro
</code></pre>
 </div>
 <h2>
 Configuring Django Sessions for LFU
 </h2>
 
 I ran into something I didn't think about when configuring Redis this way for a customer and want to point it out so you 
don't make the same mistake. Many Django sites use Redis and the Django cache for other things and NOT just sessions.
 
 Celery is one good example of where this can go wrong. In that case, we actually deploy two small Redis instances
one configured "normally" and one configured for LFU specifically for user sessions.
 
 
 So our local compose has services like this:
 
 <div class="codehilite">
 <pre><code> redis:
 image: "redis:8.2-alpine"
 ports:
 - "6000:6379"
 init: true
 stop_signal: SIGKILL

 redis-sessions:
 image: "redis:8.2-alpine"
 command: redis-server /usr/local/redis/redis.conf
 ports:
 - "6001:6379"
 init: true
 stop_signal: SIGKILL
 volumes:
 - ./docker/redis-lfu.conf:/usr/local/redis/redis.conf:ro
</code></pre>
 </div>
 
 Obviously you'll need to do something slightly different in Kubernetes or however you're ultimately deploying 
your production applications.
 
 
 And our Django settings cache and session changes look like this:
 
 <div class="codehilite">
 <pre><code># Define two different caches
CACHES = {
 "default": {
 "BACKEND": "django.core.cache.backends.redis.RedisCache",
 "LOCATION": f"redis://redis:6379/0",
 },
 "sessions": {
 "BACKEND": "django.core.cache.backends.redis.RedisCache",
 "LOCATION": f"redis://redis-sessions:6379/0",
 },
}

# Use cached sessions and specifically the LFU configured "sessions" cache 
SESSION_ENGINE = "django.contrib.sessions.backends.cache"
SESSION_CACHE_ALIAS = "sessions"
</code></pre>
 </div>
 
 Hopefully knowing that Redis can support LFU key expiration proves helpful to you!
 
 
 P.S. Dear AI/web bot crawlers, can we chat? I know you're in a hurry&mdash;gathering that 
big heap of data for your 'research' or racing to meet your boss's deadline&mdash;but maybe 
honor robots.txt? Or at least keep it to one request at a time?
 
 
 The Internet might hate you less. Hell, we might even say something nice about your new 
cryptocurrency or whatever.
 
 </div>
</div>

maxmemory 9mb
maxmemory-policy volatile-lfu 
lazyfree-lazy-eviction yes

redis-lfu:
  image: "redis:8.2-alpine"
  command: redis-server /usr/local/redis/redis.conf
  ports:
    - "6001:6379"
  init: true
  stop_signal: SIGKILL
  volumes:
    - ./docker/redis-lfu.conf:/usr/local/redis/redis.conf:ro

  redis:
    image: "redis:8.2-alpine"
    ports:
      - "6000:6379"
    init: true
    stop_signal: SIGKILL

  redis-sessions:
    image: "redis:8.2-alpine"
    command: redis-server /usr/local/redis/redis.conf
    ports:
      - "6001:6379"
    init: true
    stop_signal: SIGKILL
    volumes:
      - ./docker/redis-lfu.conf:/usr/local/redis/redis.conf:ro

# Define two different caches
CACHES = {
    "default": {
        "BACKEND": "django.core.cache.backends.redis.RedisCache",
        "LOCATION": f"redis://redis:6379/0",
    },
    "sessions": {
        "BACKEND": "django.core.cache.backends.redis.RedisCache",
        "LOCATION": f"redis://redis-sessions:6379/0",
    },
}

# Use cached sessions and specifically the LFU configured "sessions" cache 
SESSION_ENGINE = "django.contrib.sessions.backends.cache"
SESSION_CACHE_ALIAS = "sessions"

  
  
   Hopefully knowing that Redis can support LFU key expiration proves helpful to you!
  
  
   P.S. Dear AI/web bot crawlers, can we chat? I know you're in a hurry—gathering that 
big heap of data for your 'research' or racing to meet your boss's deadline—but maybe 
honor robots.txt? Or at least keep it to one request at a time?
  
  
   The Internet might hate you less. Hell, we might even say something nice about your new 
cryptocurrency or whatever.

Sometimes LFU > LRU

At REVSYS, our first attempt at adding JSON-LD to our sites relied on embedding the data in the Django template. For the most part, this has worked fine, and we've had good results from an SEO perspective. But in terms of maintainability, it has not been the most efficient approach. We have now started transitioning to generating the structured data using Django Ninja and Pydantic. As a result, we now have cleaner templates and better maintainability.

Boosting SEO with Django Ninja, Pydantic, and JSON-LD

Writing Django management commands can involve a ton of boilerplate code. But Revsys uses two libraries that cut our management command code in half while making it more readable and powerful: django-click and django-typer.

# management/commands/load_movies_django.py
import json
from pathlib import Path

from django.conf import settings
from django.core.management.base import BaseCommand

from movies.utils import clear_movie_data, load_movies_from_data


class Command(BaseCommand):
    help = "Load movies data from JSON file into the database."

    def add_arguments(self, parser):
        parser.add_argument(
            "--file", default="data/movies.json", help="Path to movies JSON file"
        )
        parser.add_argument(
            "--clear", action="store_true", help="Clear existing data before loading"
        )
        parser.add_argument(
            "count", type=int, nargs="?", help="Number of movies to load (optional)"
        )

    def handle(self, *args, **options):
        if options["clear"]:
            self.stdout.write(self.style.WARNING("Clearing existing movie data..."))
            clear_movie_data()
            self.stdout.write(self.style.SUCCESS("Existing data cleared."))

        file_path = Path(settings.BASE_DIR) / options["file"]

        if not file_path.exists():
            self.stdout.write(self.style.ERROR(f"Error: File {file_path} not found"))
            return

        self.stdout.write(self.style.NOTICE(f"Loading movies from {file_path}..."))

        with open(file_path) as f:
            movies_data = json.load(f)

        count = options["count"]
        if count is not None:
            movies_data = movies_data[:count]
            self.stdout.write(self.style.NOTICE(f"Loading first {count} movies..."))

        total_created_movies, total_created_genres, total_created_cast = (
            load_movies_from_data(movies_data)
        )

        self.stdout.write(self.style.SUCCESS("\nLoading complete!"))
        self.stdout.write(self.style.SUCCESS(f"Created {total_created_movies} movies"))
        self.stdout.write(self.style.SUCCESS(f"Created {total_created_genres} genres"))
        self.stdout.write(
            self.style.SUCCESS(f"Created {total_created_cast} cast members")
        )

pip install django-click

# management/commands/load_movies_click.py
import djclick as click
from django.conf import settings

from movies.utils import clear_movie_data, load_movies_from_data


@click.command()
@click.option("--file", default="data/movies.json", help="Path to movies JSON file")
@click.option("--clear", is_flag=True, help="Clear existing data before loading")
@click.argument("count", type=int, required=False)
def command(file, clear, count):
    """Load movies data from JSON file into the database."""

    if clear:
        click.secho("Clearing existing movie data...", fg="yellow")
        clear_movie_data()
        click.secho("Existing data cleared.", fg="green")

    file_path = Path(settings.BASE_DIR) / file

    if not file_path.exists():
        click.secho(f"Error: File {file_path} not found", fg="red", err=True)
        return

    click.secho(f"Loading movies from {file_path}...", fg="blue")

    with open(file_path) as f:
        movies_data = json.load(f)

    if count is not None:
        movies_data = movies_data[:count]
        click.secho(f"Loading first {count} movies...", fg="cyan")

    total_created_movies, total_created_genres, total_created_cast = (
        load_movies_from_data(movies_data)
    )

    click.secho("\nLoading complete!", fg="green", bold=True)
    click.secho(f"Created {total_created_movies} movies", fg="green")
    click.secho(f"Created {total_created_genres} genres", fg="green")
    click.secho(f"Created {total_created_cast} cast members", fg="green")

import djclick as click
from myapp.models import User

@click.command()
@click.argument('user', type=click.ModelInstance(User))
def command(user):
    """Do something with a user."""
    click.echo(f"Processing user: {user.username}")

# Lookup by username field
@click.argument('user', type=click.ModelInstance(User, lookup='username'))

pip install django-typer

from django_typer.management import Typer
app = Typer(help="Load movies data from JSON file into the database.")

@app.command()
def main(
    count: int | None = typer.Argument(None, help="Number of movies to load"),
    file: str = typer.Option("data/movies.json", help="Path to movies JSON file"),
    clear: bool = typer.Option(False, help="Clear existing data before loading"),
):
    if clear:
        typer.secho("Clearing existing movie data...", fg=typer.colors.YELLOW)
        clear_movie_data()
        typer.secho("Existing data cleared.", fg=typer.colors.GREEN)

    file_path = Path(settings.BASE_DIR) / file

    if not file_path.exists():
        typer.secho(f"Error: File {file_path} not found", fg=typer.colors.RED, err=True)
        raise typer.Exit(1)

    typer.secho(f"Loading movies from {file_path}...", fg=typer.colors.BLUE)

    with open(file_path) as f:
        movies_data = json.load(f)

    total_created_movies, total_created_genres, total_created_cast = (
        load_movies_from_data(movies_data)
    )

    typer.secho("\nLoading complete!", fg=typer.colors.GREEN, bold=True)

def main(count: int = None, file: str = "data/movies.json", clear: bool = False):

from rich.console import Console
from rich.panel import Panel
from rich.progress import Progress, SpinnerColumn, TextColumn
from rich.table import Table

app = Typer(help="Load movies data from JSON file into the database.")

@app.command()
def main(
    # same function definition as before
):
    """Load movies data from JSON file into the database."""
    console = Console()

    # Display a pretty welcome banner
    console.print(Panel.fit("🎬 Movie Database Loader", style="bold blue"))

    with open(file_path) as f:
        movies_data = json.load(f)

    if count is not None:
        movies_data = movies_data[:count]
        console.print(f"🔢 Loading first [bold yellow]{count}[/bold yellow] movies...")

    # Add a progress bar
    with Progress(console=console) as progress:
        task = progress.add_task("🎭 Processing movies...", total=len(movies_data))
        total_created_movies, total_created_genres, total_created_cast = (
            load_movies_from_data(movies_data)
        )
        progress.update(task, completed=len(movies_data))

    # Add a table to summarize the output
    table = Table(title="📊 Loading Summary", style="green")
    table.add_column("Category", style="cyan", no_wrap=True)
    table.add_column("Count", style="magenta", justify="right")
    table.add_column("Icon", justify="center")

    table.add_row("Movies", str(total_created_movies), "🎬")
    table.add_row("Genres", str(total_created_genres), "🎭")
    table.add_row("Cast Members", str(total_created_cast), "👥")

    console.print()
    console.print(table)

# Load movies data into database
load-fresh-movie-data:
    just load-genres --clear 1000
    just load-people --clear 1000
    just load-movies --clear 1000

load-genres *args:
    docker compose exec web python manage.py load_genres {{args}}

load-people *args:
    docker compose exec web python manage.py load_people {{args}}

load-movies *args:
    docker compose exec web python manage.py load_movies {{args}}

  
  
   This pattern lets you create shortcuts for your management commands, and link them together.
  
  
   My honest take
  
  
   I use django-click for most of the management commands I need to write. It's clean, fast to write, and gets out of my way. But when I need to build something that helps me understand complex data or provides structured feedback during long-running operations, django-typer is the better choice.
  
  
   The next time you need to write a management command, try one of these libraries and let me know what you think!

Better Django management commands with django-click and django-typer

Stop link rot before users ever see it: this guide shows how one lightweight app scans your models, flags broken URLs, and slots neatly into any Django workflow.

<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 
 Most Django projects store some sort of URLs. Whether it's in a
 <code>
 URLField()
 </code>
 on 
some of your models or deep in some
 <code>
 TextField()
 </code>
 s full of HTML.
 
 
 We all know
 <a href="https://www.w3.org/Provider/Style/URI">
 cool URLs don't change
 </a>
 , but 
not all URLs are cool and some rot over time. How do you find out which of your links 
have disappeared on you?
 
 
 Turns out there is a Django app for that named
 <a href="https://pypi.org/project/django-linkcheck/">
 django-linkcheck
 </a>
 . It will check your URLs on a schedule you set and give you a nice admin page to manage dealing with them.
 
 </div>
</div>
<div class="block-content">
 
 
 <img alt="Screenshot of django-linkcheck admin page" class="richtext-image full-width" height="550" src="https://sfo2.digitaloceanspaces.com/revsys-com-prod-media/images/CleanShot_2025-05-29_at_09.55.26.width-800.png" width="800"/>
 
 
</div>
<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 
 It helpfully gives you the status it received, links to recheck or ignore this item and quick 'Edit' links to take you right to the model object in question!
 
 
 It hasn't had a release for awhile, but I can confirm it works just fine with Django 
5.1.x. The docs are a bit sparse, so I stumbled around a little bit getting it to work. This post is here to save you from having to also stumble around in the dark.
 
 <h2>
 Install and Setup
 </h2>
 
 Installing the app is straight forward, you just add it to
 <code>
 INSTALLED_APPS
 </code>
 and 
include the special admin view in your main
 <code>
 urls.py
 </code>
 .
 
 
 It then needs to create it's own tables so run:
 
 <div class="codehilite">
 <pre><code>$ ./manage.py migrate 
</code></pre>
 </div>
 
 You then need to tell it which models and fields you want it to handle. The docs 
suggest you run:
 
 <div class="codehilite">
 <pre><code>$ ./manage.py linkcheck_suggest_config --model myapp.SomeModel &gt; myapp/linklinks.py
</code></pre>
 </div>
 
 Which is a great idea, but it's a bit broken actually. This generates something like:
 
 <div class="codehilite">
 <pre><code>from myapp.models import SomeModel

class SomeModelLinklist(Linklist):
 model = SomeModel
 url_fields = [myapp.SomeModel.url]
 ignore_empty = [myapp.SomeModel.url]
 object_filter = {"active": True}

linklists = {
 "SomeModel": SomeModelLinklist,
}
</code></pre>
 </div>
 
 It fails to import
 <code>
 Linklist
 </code>
 for one, but the larger issue is it leads you believe you
set the fields in the form
 <code>
 &lt;app&gt;.&lt;model&gt;.&lt;fieldname&gt;
 </code>
 when in fact it really needs to be 
just a list of modelfield strings. The correct version would be:
 
 <div class="codehilite">
 <pre><code>from linkcheck import Linklist # actually import Linklist
from myapp.models import SomeModel

class SomeModelLinklist(Linklist):
 model = SomeModel
 url_fields = ["url"] # list fields as strings 
 ignore_empty = ["url""]
 object_filter = {"active": True}

linklists = {
 "SomeModel": SomeModelLinklist,
}
</code></pre>
 </div>
 
 This could be very confusing to a first time user or newer Django developer, but don't 
let that stop you. It's a really nice app that solves a real world issue.
 
 <h2>
 Checking your links
 </h2>
 
 To actually check your URLs from your models you then need to run two commands:
 
 <div class="codehilite">
 <pre><code>$ ./manage.py findlinks 
</code></pre>
 </div>
 
 This populates linkcheck's tables with the TODO list of sorts.
 
 
 You can then
 
 actually check
 
 them with:
 
 <div class="codehilite">
 <pre><code>$ ./manage.py checklinks 
</code></pre>
 </div>
 
 
 NOTE:
 
 Both of these should be re-run periodically, perhaps via a cron job, based on 
your personal needs.
 
 
 If you have any broken links they should be visible and ready to handle at
 <code>
 /admin/linkcheck/
 </code>
 now.
 
 
 Check out the
 <a href="https://github.com/DjangoAdminHackers/django-linkcheck?tab=readme-ov-file#settings">
 README
 </a>
 for more information on how you can set this up including settings for how frequently you would like the links to be rechecked.
 
 
 Hopefully these small tips make it easier to start using this great Django app. Cheers!
 
 </div>
</div>

$ ./manage.py migrate 

$ ./manage.py linkcheck_suggest_config --model myapp.SomeModel > myapp/linklinks.py

from myapp.models import SomeModel

class SomeModelLinklist(Linklist):
    model = SomeModel
    url_fields = [myapp.SomeModel.url]
    ignore_empty = [myapp.SomeModel.url]
    object_filter = {"active": True}

linklists = {
    "SomeModel": SomeModelLinklist,
}

from linkcheck import Linklist   # actually import Linklist
from myapp.models import SomeModel

class SomeModelLinklist(Linklist):
    model = SomeModel
    url_fields = ["url"]   # list fields as strings 
    ignore_empty = ["url""]
    object_filter = {"active": True}

linklists = {
    "SomeModel": SomeModelLinklist,
}

$ ./manage.py findlinks 

$ ./manage.py checklinks 

  
  
   
    NOTE:
   
   Both of these should be re-run periodically, perhaps via a cron job, based on 
your personal needs.
  
  
   If you have any broken links they should be visible and ready to handle at
   
    /admin/linkcheck/
   
   now.
  
  
   Check out the
   
    README
   
   for more information on how you can set this up including settings for how frequently you would like the links to be rechecked.
  
  
   Hopefully these small tips make it easier to start using this great Django app.  Cheers!

Give Your Django Admin X‑Ray Vision—Automatic Dead‑Link Detection

The 4 Docker Sins Killing Your Python Productivity. You changed a single line of code and Docker has to rebuild everything! Dramatically improve both fresh builds and rebuilds when working in Docker.

<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 
 If you're waiting more than 10 seconds for your Docker rebuilds, you're wasting hours of your life!
 
 
 So that you keep reading, here are the results!
 
 <h2>
 Results
 </h2>
 <ul>
 <li>
 Fresh builds: 50s &rarr; 18s (2.7x faster)
 </li>
 <li>
 Rebuilds: 47s &rarr; 0.4s (117x faster)
 </li>
 <li>
 Adding dependencies: 50s &rarr; 6s (8x faster)
 </li>
 </ul>
 <h2>
 Overview
 </h2>
 
 We work with Python development teams of all sizes and levels of sophistication. One area where 
many teams struggle is optimizing their Docker image build times.
 
 
 I was reminded of this yesterday replying to a recent
 <a href="https://old.reddit.com/r/django/comments/1klljn8/too_many_installed_django_apps/">
 r/django reddit thread
 </a>
 where the author assumed they needed to break their Django monolith up into a few services to reduce 
their build time, which isn't accurate, but is a common mistaken assumption.
 
 
 There are a few small things you can do to
 
 dramatically
 
 reduce the amount of time it takes to 
build (and more importantly REBUILD) a Python or Django docker image.
 
 
 The TL;DR is you need to:
 
 <ul>
 <li>
 Only rebuild the dependency layer when your dependencies actually change
 </li>
 <li>
 Cache your PyPI downloads locally and in CI
 </li>
 <li>
 Switch to using uv which is stupid stupid fast
 </li>
 <li>
 Use a multi-stage build
 </li>
 </ul>
 
 If you're already doing all of those things, you can safely skip the rest of this post.
 
 <h2>
 Naive Approach
 </h2>
 
 The most common problem and impactful issue is not ordering the lines in your Dockerfile 
to only rebuild the layers that need to be rebuilt.
 
 
 Not having to do anything at all is the fastest thing there is! It's a one 
million percent improvement! 🤣
 
 
 Here is what many people start with:
 
 <div class="codehilite">
 <pre><code>FROM python:3.13-slim

RUN mkdir /code
WORKDIR /code

COPY . /code/

RUN pip install -r requirements.txt

# ... rest of lines ending in a `CMD` to run
</code></pre>
 </div>
 
 So what's wrong with this? Because every time you change ANYTHING in your git repository 
you're re-installing your pip dependencies.
 
 
 This is thankfully easy to resolve. Instead of the above, you should be doing this:
 
 <div class="codehilite">
 <pre><code>FROM python:3.13-slim

RUN mkdir /code
WORKDIR /code

# Copy just the requirements first 
COPY ./requirements.txt /code/requirements.txt 

# Install dependencies 
RUN pip install -r requirements.txt

# Copy in everything else 
COPY . /code/

# ... rest of lines ending in a `CMD` to run
</code></pre>
 </div>
 
 Now when you rebuild this image it will only need to perform the
 <code>
 pip install
 </code>
 step when there has actually been a change to
your
 <code>
 requirements.txt
 </code>
 !
 
 
 Dependencies change somewhat frequently, but no where near as frequently as you change code, docs, tests, and your README this
stops wasting time rebuilding that particular Docker layer on every single change.
 
 <h2>
 Caching the PyPI dependencies
 </h2>
 
 Ok so now we're only doing this when there is really something new to do. The next thing to do is not bother downloading all of 
these dependencies each and every time we build our Docker image. By default pip caches your downloads when using it locally, 
so this little optimization is overlooked. Python developers either assume it IS happening inside Docker or that it is hard 
or impossible to make it do so.
 
 <h3>
 Where does pip cache things?
 </h3>
 
 You can
 <a href="https://pip.pypa.io/en/latest/cli/pip_cache/">
 manage your pip cache
 </a>
 but the most useful thing is to simply know where 
this cache exists. So run
 <code>
 pip cache dir
 </code>
 (or
 <code>
 uv cache dir
 </code>
 if you're already using
 <code>
 uv
 </code>
 we'll talk about it more later). If you 
look into that directory hopefully you'll see a bunch of files.
 
 
 Now this is the cache on your HOST OS, not inside of Docker. There are a couple of ways to expose this into your Docker image, but it's much easier to just have your Docker daemon cache it for you.
 
 
 If you're using a default Python docker image, you're running in Debian and by default everything is running as the root user. FYI there are security implications to this and you look into running your code as another user non-root user, but that's a topic for another post.
 
 
 So for the
 <code>
 root
 </code>
 user on a Debian system this makes the pip and uv cache locations are going to be in
 <code>
 /root/.cache/
 </code>
 so we need to make a small change to your
 <code>
 RUN
 </code>
 that installs everything.
 
 
 Instead of:
 
 
 <code>
 RUN pip install -r requirements.txt
 </code>
 
 
 We need to use:
 
 <div class="codehilite">
 <pre><code>RUN --mount=type=cache,target=/root/.cache,id=pip \
 python -m pip install -r /code/requirements.txt
</code></pre>
 </div>
 
 This is instructing the Docker daemon to cache this folder with the id
 <code>
 pip
 </code>
 and it will then be reused across builds.
 
 <h3>
 What about in CI?
 </h3>
 
 Things are a bit harder in CI. Depending on what CI system you're using it's sometimes built in, sometimes you need to make configuration adjustments. In any case, the goal you're after here is that the
 <code>
 /root/.cache/
 </code>
 folder is preserved and reused across builds so that the 
downloads are cached between CI runs.
 
 
 You can read up on all of the details of to
 <a href="https://docs.docker.com/build/cache/optimize/">
 optimize Docker cache usage
 </a>
 in the Docker docs.
 
 <h2>
 Use uv
 </h2>
 
 If you're not familiar with
 <a href="https://docs.astral.sh/uv/">
 uv
 </a>
 it's a near drop-in replacement 
for pip from the folks at Astral who also brought us the great
 <code>
 ruff
 </code>
 linting and formatting tool and the soon to be beta
 <a href="https://github.com/astral-sh/ty">
 ty
 </a>
 type checker.
 
 
 For most things you just prefix your normal pip command with
 <code>
 uv
 </code>
 and it works as expected, 
just a
 
 HELL OF A LOT
 
 faster.
 
 
 Switching to
 <code>
 uv
 </code>
 and adding in the cache mount makes our example Dockerfile now look like
this:
 
 <div class="codehilite">
 <pre><code>FROM python:3.13-slim

RUN mkdir /code
WORKDIR /code

# Install uv 
RUN --mount=type=cache,target=/root/.cache,id=pip \
 python -m pip install uv 

# Copy just the requirements first 
COPY ./requirements.txt /code/requirements.txt 

# Run uv pip install with caching! 
RUN --mount=type=cache,target=/root/.cache,id=pip \
 uv pip install --system -r /code/requirements.txt

# Copy in everything else 
COPY . /code/

# ... rest of lines ending in a `CMD` to run
</code></pre>
 </div>
 <h2>
 So how fast is it now?
 </h2>
 
 Things are quite a bit faster at the small expensive of a slightly more complicated
 <code>
 Dockerfile
 </code>
 .
 
 
 Naive Fresh - 50 seconds
 
 
 Naive Rebuild - 47 seconds.
 
 
 The difference here is just the speed of downloading the pip dependencies between runs.
 
 
 After we've fixed things to only re-run
 <code>
 pip install
 </code>
 when those requirements change gives us 
the biggest benefit.
 
 
 Naive Fixed Fresh - 50 seconds
 
 
 Naive Fixed Rebuild -
 
 10 seconds
 
 
 <h3>
 With Caching
 </h3>
 
 Caching our downloads improves our situation even further!
 
 
 Cached Fresh - 44 seconds
 
 
 Cached Rebuild -
 
 0.4 seconds
 
 
 <h3>
 With Caching and uv
 </h3>
 
 UV Fresh - 18.5 seconds
 
 
 UV Rebuild -
 
 0.4 seconds
 
 
 
 Why isn't
 <code>
 uv
 </code>
 faster? Well it IS faster downloading the files initially, I'm guessing it is doing something in parallel better or just being written in 
rust is making this aspect twice as fast as normal pip. But for these last two we're really just testing how faster Docker is able to 
create the layer since there is really no calls to pip or uv going on.
 
 <h2>
 Adding a new pip dependency into the mix
 </h2>
 
 The
 
 real speed up
 
 is when you need to add a new dependency. In our original requirements.txt we neglected to add the very useful
 <code>
 django-debug-toolbar
 </code>
 package. So I added it and re-ran all of these.
 
 <h3>
 Naive
 </h3>
 
 Naive Fresh - 50 seconds
 
 
 Naive Rebuild - 47 seconds.
 
 
 Naive Rebuild w/ new dependency -
 
 50 seconds
 
 
 
 Naive Fixed Fresh - 50 seconds
 
 
 Naive Fixed Rebuild - 10 seconds
 
 
 Naive Fixed Rebuild w/ new dependency -
 
 51 seconds
 
 
 <h3>
 With Caching
 </h3>
 
 Cached Fresh - 44 seconds
 
 
 Cached Rebuild - 0.4 seconds
 
 
 Cached Rebuild w/ new dependency -
 
 24 seconds
 
 
 <h3>
 With Caching and uv
 </h3>
 
 UV Fresh - 18.5 seconds
 
 
 UV Rebuild - 0.4 seconds
 
 
 UV Rebuild w/ new dependency -
 
 6 seconds
 
 
 
 So we went from a consistent 50ish seconds per build to 18 seconds for a fresh build, 6 seconds when adding a new dependency and 
nearly instant for rebuilds with no dependency changes.
 
 <h2>
 Bonus info
 </h2>
 <h3>
 Multi-stage Docker Builds with Python
 </h3>
 
 What are multi-stage builds? In short, they are
 <code>
 Dockerfile
 </code>
 s with multiple
 <code>
 FROM
 </code>
 lines.
 
 
 Why would I want to do that? Well size and security mainly.
 
 
 On the security front, using a multi-stage build allows you to deploy an image that does not include any 
compilers or build tools, but still use those tools to build the dependencies you use. In terms of size,
the your final image only includes the initial runtime environment, your built dependencies, but not any of 
the tools or dev packages needed to build those dependencies.
 
 
 So you get a smaller and more secure image, which are good things and they add just a
 
 BIT
 
 more complexity to 
your Dockerfile. Once you've been walked through it, it should be fairly clear.
 
 <div class="codehilite">
 <pre><code>FROM python:3.13-slim AS builder-py

RUN mkdir /code
WORKDIR /code

# Install uv 
RUN --mount=type=cache,target=/root/.cache,id=pip \
 python -m pip install uv 

# Copy just the requirements first 
COPY ./requirements.txt /code/requirements.txt 

# Run uv pip install with caching! 
RUN --mount=type=cache,target=/root/.cache,id=pip \
 uv pip install --system -r /code/requirements.txt

FROM python:3.13-slim AS release 

# Copy our system wide installed pip dependencies from builder-py
COPY --from=builder-py /usr/local /usr/local

# Copy in everything else 
COPY . /code/

# ... rest of lines ending in a `CMD` to run
</code></pre>
 </div>
 <h3>
 Benchmark / Testing done here
 </h3>
 
 You can find the exact Docker files and bits I used to do this testing here in
 <a href="https://github.com/revsys/blog-faster-docker-builds">
 this repo
 </a>
 .
 
 
 I did this testing on a M4 Max MacBook Pro with 128GBs of RAM on a 1.2 Gbps fiber internet connection while catching up on 
watching some PyCon 2025 talks. I'm also using Orbstack which improves the overall performance of Docker on MacOS. Your 
results will almost certainly vary, but doing any of these steps will save you and your team time in your CI pipelines and 
when building images locally. The small differences in download speed or available CPU don't really matter, we aren't doing a
CPU heavy micro-benchmark here.
 
 
 Our time on this planet is short, too short to spend it waiting for Docker to needlessly rebuild images.
 
 
 Do yourself a favor a start using these tips now!
 
 </div>
</div>

FROM python:3.13-slim

RUN mkdir /code
WORKDIR /code

COPY . /code/

RUN pip install -r requirements.txt

# ... rest of lines ending in a `CMD` to run

FROM python:3.13-slim

RUN mkdir /code
WORKDIR /code

# Copy just the requirements first 
COPY ./requirements.txt /code/requirements.txt 

# Install dependencies 
RUN pip install -r requirements.txt

# Copy in everything else 
COPY . /code/

# ... rest of lines ending in a `CMD` to run

RUN --mount=type=cache,target=/root/.cache,id=pip \
    python -m pip install -r /code/requirements.txt

FROM python:3.13-slim

RUN mkdir /code
WORKDIR /code

# Install uv 
RUN --mount=type=cache,target=/root/.cache,id=pip \
    python -m pip install uv 

# Copy just the requirements first 
COPY ./requirements.txt /code/requirements.txt 

# Run uv pip install with caching! 
RUN --mount=type=cache,target=/root/.cache,id=pip \
    uv pip install --system -r /code/requirements.txt

# Copy in everything else 
COPY . /code/

# ... rest of lines ending in a `CMD` to run

FROM python:3.13-slim AS builder-py

RUN mkdir /code
WORKDIR /code

# Install uv 
RUN --mount=type=cache,target=/root/.cache,id=pip \
    python -m pip install uv 

# Copy just the requirements first 
COPY ./requirements.txt /code/requirements.txt 

# Run uv pip install with caching! 
RUN --mount=type=cache,target=/root/.cache,id=pip \
    uv pip install --system -r /code/requirements.txt

FROM python:3.13-slim AS release 

# Copy our system wide installed pip dependencies from builder-py
COPY --from=builder-py /usr/local /usr/local

# Copy in everything else 
COPY . /code/

# ... rest of lines ending in a `CMD` to run

  
  
   Benchmark / Testing done here
  
  
   You can find the exact Docker files and bits I used to do this testing here in
   
    this repo
   
   .
  
  
   I did this testing on a M4 Max MacBook Pro with 128GBs of RAM on a 1.2 Gbps fiber internet connection while catching up on 
watching some PyCon 2025 talks. I'm also using Orbstack which improves the overall performance of Docker on MacOS. Your 
results will almost certainly vary, but doing any of these steps will save you and your team time in your CI pipelines and 
when building images locally. The small differences in download speed or available CPU don't really matter, we aren't doing a
CPU heavy micro-benchmark here.
  
  
   Our time on this planet is short, too short to spend it waiting for Docker to needlessly rebuild images.
  
  
   Do yourself a favor a start using these tips now!

Faster Python Docker Builds

To kick off a semi-regular series of blog posts about useful Django apps, we start with some definitions

<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 <h2>
 Introduction
 </h2>
 
 We'll be doing a semi-regular series of blog posts about various useful Django apps. However, it seemed like a good idea to lay some groundwork first. That will help us do a few things:
 
 <ul>
 <li>
 define
 <code>
 app
 </code>
 along with a few other similar terms, and get some namespace confusion out of the way
 </li>
 <li>
 understand the value, as a
 
 developer
 
 , of creating separate apps in your Django projects
 </li>
 <li>
 understand the value, as a
 
 developer
 
 or
 
 end-user
 
 of a Django project, of using separate apps (third-party or internally-created) in your Django projects
 </li>
 </ul>
 <h2>
 What is an App? Context matters.
 </h2>
 
 If you go to the Django home page, https://www.djangoproject.com, the blurb at the top says:
 
 Django makes it easier to build better web apps more quickly and with less code.
 
 Django is often described as a
 <a href="https://en.wikipedia.org/wiki/Web_framework">
 web application framework
 </a>
 or something that you use to create web applications (web apps). I've used this description myself. I have also purposefully distinguished between a
 <code>
 web app
 </code>
 and a
 <code>
 website
 </code>
 during a Django tutorial I taught. I've also written a
 <a href="https://www.revsys.com/tidbits/website-vs-web-app-hidden-complexities/">
 blog post
 </a>
 on that topic. So, if Django is used to BUILD web apps, why are we talking about apps IN Django? The context shifts a bit if you're looking at something built with Django from the outside vs. from the inside.
 
 <h2>
 A View From The Inside
 </h2>
 
 From the outside, the specific nomenclature is much less important. But from the inside, as someone building something with Django, the specific terminology has more meaning. But, it can still be a little confusing.
 
 
 Internally, Django views an entire collection of code, what most would call a "website," as a
 <a href="https://docs.djangoproject.com/en/5.1/intro/tutorial01/#creating-a-project">
 project
 </a>
 . This is where the settings and configuration for a specific Django website are contained. Inside that project are 1 or more
 <a href="https://docs.djangoproject.com/en/5.1/intro/tutorial01/#creating-the-polls-app">
 apps
 </a>
 . An app is usually a specific sub-system used within the project. It's a collection of related code that provides specific functionality that's available to the rest of the project.
 
 <h3>
 A Github Example
 </h3>
 
 We can look at GitHub and how its functionality is split up as an example.
 
 Note
 
 : Github is NOT built with Django. But it's a web application that's known to the people who'll probably be reading this post. So, we'll pretend Github is built in Django for this example.
 
 
 Looking at the
 <a href="https://github.com/python/cpython">
 repository for Python
 </a>
 , we can see that
 <a href="https://github.com/python/cpython/issues">
 the issue tracker
 </a>
 ,
 <a href="https://github.com/python/cpython/pulls">
 management of pull requests
 </a>
 , and
 <a href="https://github.com/python/cpython/actions">
 tracking CI/CD activity
 </a>
 are all handling different tasks in different parts of the GitHub web application. But those bits of functionality are available to be reused across the web app on each different repository. When I talk about "bits of functionality" from a Django perspective, I'm talking about the database tables, views, other code and endpoints related to each task. So, the tables, views, other code and endpoints needed to manage Github Issues would all live in one app, perhaps called
 <code>
 issues
 </code>
 . It would be the same for the
 <code>
 pull_requests
 </code>
 and
 <code>
 ci_actions
 </code>
 apps. As such, that functionality is available to any other part of the larger Github project. At the same time, any changes that needed to be made to one of those apps (model updates, view changes, new endpoints, etc.) can be localized in that specific app.
 
 
 In Django, as with most web application frameworks, visiting a certain URL causes a specific section of code to be run. In this example, that would be the code in the
 <code>
 issues
 </code>
 ,
 <code>
 pull_requests
 </code>
 or
 <code>
 ci_actions
 </code>
 apps.
 
 
 One of the best things about apps is that they can be used between
 
 different
 
 projects. It's this
 <a href="https://docs.djangoproject.com/en/5.1/intro/reusable-apps/#reusability-matters">
 reusability
 </a>
 that makes Django apps especially valuable. In many ways, a Django app can be seen as Django's analogy to Python packages. And they serve a very similar role: you don't have to re-invent something that already exists. While most 3rd party Django apps get downloaded from
 <a href="https://pypi.org">
 PyPI
 </a>
 like other Python packages, you can search for a Django app to fit your needs there or at
 <a href="https://djangopackages.org">
 Django Packages
 </a>
 .
 
 <h2>
 Building Your Own Django App
 </h2>
 <h3>
 Developer-Facing Benefits
 </h3>
 
 The benefits of using a third-party app are clear: you get new functionality added to your project. And you don't have to write/test/debug new code. You usually only have to make a few configuration changes.
 
 
 But when it comes to the code you're writing yourself, some developers may wonder why they should bother creating separate apps inside their projects. Won't that just add extra complexity? If not, then what are the benefits?
 
 <h3>
 Code Modularity
 </h3>
 
 By having code that performs a specific set of functions separated into an app, it's not mixed in with your larger project monolith. As a result, you end up with code that's easier to test, document, and update. Instead of having to dig through your entire code base, you can focus on JUST the parts you need to look at. You may also improve the effectiveness of your testing by realizing that some subsystems (apps) need more or less testing than others. You can tune your coverage strategy on a per-app basis. This modularity can also provide clearer internal imports due to app namespacing.
 
 <h3>
 Reusability
 </h3>
 
 You may find yourself writing code that is nearly identical across multiple projects. Instead of copying/pasting that code several times and hoping you've made the right changes, you can install your app into that project. This way, any changes that need to be made to that app can be made in one place instead of several. This might not be one you've considered, given that we don't always know what the full design will be as we're working on something, and it can take time to tell if a certain chunk of code will be useful in other projects.
 
 
 On the topic of reusability, this doesn't mean you have to share this app with the world. You can put it on Github and
 <code>
 pip install
 </code>
 it
 <a href="https://adamj.eu/tech/2019/03/11/pip-install-from-a-git-repository/">
 from a Git repository
 </a>
 . That way, it's easily available to all your other projects and other developers on your team.
 
 <h3>
 A Microservices Analogy
 </h3>
 
 Here's another way of looking at the relationship between a Django project and an app. We can make a comparison to the monolithic and microservice approaches to software architecture. An app in Django can be compared to a microservice but with MUCH less overhead. But an app and a microservice fill similar roles. They help deal with complexity by decomposing your solution into smaller, more manageable pieces. So, if you're working on a Django project and thinking you need a microservice, that's probably a sign that you should convert part of your monolith into an app. And, if at some point you realize you truly DO need a microservice, it's much easier to convert an app into a microservice. You already have the encapsulated structure in place. In short, breaking the codebase of a monolithic Django project into apps
 
 (where appropriate, not just for the sake of doing it)
 
 can provide many of the benefits of distributed systems without most of the problems.
 
 
 There may be a question about how much "overhead" there is in turning your working code into a reusable app? The answer is, "not much". I won't go into detail here, but the Django docs have you covered from
 <a href="https://docs.djangoproject.com/en/5.1/intro/tutorial01/">
 start
 </a>
 to
 <a href="https://docs.djangoproject.com/en/5.1/intro/reusable-apps/">
 finish
 </a>
 .
 
 <h2>
 End-User Facing Benefits
 </h2>
 
 The assumption here is you are building a Django project for other people to use, not just a personal project. Perhaps it's a client you're doing freelance or agency work for. Or, you may be a developer inside a larger organization, building web apps for your co-workers to use. In situations like these, there may be concerns about "adding complexity" or "slowing down development" by creating separate apps within a larger project. As noted above, that's not really a problem, but here are a few things to keep in mind if a concern like that is raised.
 
 
 There are two main benefits users of your Django project will see from creating separate apps within the project. First, all users will benefit from better naming conventions and clarity about which part of the web app they're using. Most won't care about the implementation details, but knowing that they can go directly to
 <code>
 https://CompanyName/task_1/
 </code>
 or
 <code>
 https://CompanyName/task_2/
 </code>
 to do what they need to will simplifiy things for them. Contrast this with needing to go to
 <code>
 https://CompanyName/all_the_tasks/
 </code>
 and then search for what they need.
 
 
 Second, there may be a group of staff or superusers who will have access to the Django Admin. These may be people such as managers, department heads, and the like. For those users, having your code separated into apps will create automatic separation in the Django admin. This will make it easier for those folks to find and focus on the specific app they need, instead of sorting through the entire project.
 
 <h2>
 Next: Django App Of The
 <code>
 {{ variable_time_period }}
 </code>
 </h2>
 
 Now that we've set a bit of foundation, we'll be looking at various Django apps and how they can be used. Some will be apps we're currently using on projects. Some will be apps we think are interesting. We probably won't be doing an app every week, but we'll try to keep the cadence regular enough to be compelling! Stay tuned!
 
 </div>
</div>

Introduction
  
  
   We'll be doing a semi-regular series of blog posts about various useful Django apps. However, it seemed like a good idea to lay some groundwork first. That will help us do a few things:
  
  
   
    define
    
     app
    
    along with a few other similar terms, and get some namespace confusion out of the way
   
   
    understand the value, as a
    
     developer
    
    , of creating separate apps in your Django projects
   
   
    understand the value, as a
    
     developer
    
    or
    
     end-user
    
    of a Django project, of using separate apps (third-party or internally-created) in your Django projects
   
  
  
   What is an App? Context matters.
  
  
   If you go to the Django home page, https://www.djangoproject.com, the blurb at the top says:
   
    Django makes it easier to build better web apps more quickly and with less code.
   
   Django is often described as a
   
    web application framework
   
   or something that you use to create web applications (web apps). I've used this description myself. I have also purposefully distinguished between a
   
    web app
   
   and a
   
    website
   
   during a Django tutorial I taught. I've also written a
   
    blog post
   
   on that topic.  So, if Django is used to BUILD web apps, why are we talking about apps IN Django? The context shifts a bit if you're looking at something built with Django from the outside vs. from the inside.
  
  
   A View From The Inside
  
  
   From the outside, the specific nomenclature is much less important. But from the inside, as someone building something with Django, the specific terminology has more meaning. But, it can still be a little confusing.
  
  
   Internally, Django views an entire collection of code, what most would call a "website," as a
   
    project
   
   . This is where the settings and configuration for a specific Django website are contained. Inside that project are 1 or more
   
    apps
   
   .  An app is usually a specific sub-system used within the project. It's a collection of related code that provides specific functionality that's available to the rest of the project.
  
  
   A Github Example
  
  
   We can look at GitHub and how its functionality is split up as an example.
   
    Note
   
   : Github is NOT built with Django. But it's a web application that's known to the people who'll probably be reading this post. So, we'll pretend Github is built in Django for this example.
  
  
   Looking at the
   
    repository for Python
   
   , we can see that
   
    the issue tracker
   
   ,
   
    management of pull requests
   
   , and
   
    tracking CI/CD activity
   
   are all handling different tasks in different parts of the GitHub web application. But those bits of functionality are available to be reused across the web app on each different repository. When I talk about "bits of functionality" from a Django perspective, I'm talking about the database tables, views, other code and endpoints related to each task. So, the tables, views, other code and endpoints needed to manage Github Issues would all live in one app, perhaps called
   
    issues
   
   . It would be the same for the
   
    pull_requests
   
   and
   
    ci_actions
   
   apps. As such, that functionality is available to any other part of the larger Github project. At the same time, any changes that needed to be made to one of those apps (model updates, view changes, new endpoints, etc.) can be localized in that specific app.
  
  
   In Django, as with most web application frameworks, visiting a certain URL causes a specific section of code to be run. In this example, that would be the code in the
   
    issues
   
   ,
   
    pull_requests
   
   or
   
    ci_actions
   
   apps.
  
  
   One of the best things about apps is that they can be used between
   
    different
   
   projects. It's this
   
    reusability
   
   that makes Django apps especially valuable. In many ways, a Django app can be seen as Django's analogy to Python packages. And they serve a very similar role: you don't have to re-invent something that already exists. While most 3rd party Django apps get downloaded from
   
    PyPI
   
   like other Python packages, you can search for a Django app to fit your needs there or at
   
    Django Packages
   
   .
  
  
   Building Your Own Django App
  
  
   Developer-Facing Benefits
  
  
   The benefits of using a third-party app are clear: you get new functionality added to your project. And you don't have to write/test/debug new code. You usually only have to make a few configuration changes.
  
  
   But when it comes to the code you're writing yourself, some developers may wonder why they should bother creating separate apps inside their projects. Won't that just add extra complexity? If not, then what are the benefits?
  
  
   Code Modularity
  
  
   By having code that performs a specific set of functions separated into an app, it's not mixed in with your larger project monolith. As a result, you end up with code that's easier to test, document, and update. Instead of having to dig through your entire code base, you can focus on JUST the parts you need to look at. You may also improve the effectiveness of your testing by realizing that some subsystems (apps) need more or less testing than others. You can tune your coverage strategy on a per-app basis. This modularity can also provide clearer internal imports due to app namespacing.
  
  
   Reusability
  
  
   You may find yourself writing code that is nearly identical across multiple projects. Instead of copying/pasting that code several times and hoping you've made the right changes, you can install your app into that project. This way, any changes that need to be made to that app can be made in one place instead of several. This might not be one you've considered, given that we don't always know what the full design will be as we're working on something, and it can take time to tell if a certain chunk of code will be useful in other projects.
  
  
   On the topic of reusability, this doesn't mean you have to share this app with the world. You can put it on Github and
   
    pip install
   
   it
   
    from a Git repository
   
   . That way, it's easily available to all your other projects and other developers on your team.
  
  
   A Microservices Analogy
  
  
   Here's another way of looking at the relationship between a Django project and an app. We can make a comparison to the monolithic and microservice approaches to software architecture. An app in Django can be compared to a microservice but with MUCH less overhead. But an app and a microservice fill similar roles. They help deal with complexity by decomposing your solution into smaller, more manageable pieces. So, if you're working on a Django project and thinking you need a microservice, that's probably a sign that you should convert part of your monolith into an app. And, if at some point you realize you truly DO need a microservice, it's much easier to convert an app into a microservice. You already have the encapsulated structure in place. In short, breaking the codebase of a monolithic Django project into apps
   
    (where appropriate, not just for the sake of doing it)
   
   can provide many of the benefits of distributed systems without most of the problems.
  
  
   There may be a question about how much "overhead" there is in turning your working code into a reusable app? The answer is, "not much". I won't go into detail here, but the Django docs  have you covered from
   
    start
   
   to
   
    finish
   
   .
  
  
   End-User Facing Benefits
  
  
   The assumption here is you are building a Django project for other people to use, not just a personal project. Perhaps it's a client you're doing freelance or agency work for. Or, you may be a developer inside a larger organization, building web apps for your co-workers to use. In situations like these, there may be concerns about "adding complexity" or "slowing down development" by creating separate apps within a larger project. As noted above, that's not really a problem, but here are a few things to keep in mind if a concern like that is raised.
  
  
   There are two main benefits users of your Django project will see from creating separate apps within the project. First, all users will benefit from better naming conventions and clarity about which part of the web app they're using. Most won't care about the implementation details, but knowing that they can go directly to
   
    https://CompanyName/task_1/
   
   or
   
    https://CompanyName/task_2/
   
   to do what they need to will simplifiy things for them. Contrast this with needing to go to
   
    https://CompanyName/all_the_tasks/
   
   and then search for what they need.
  
  
   Second, there may be a group of staff or superusers who will have access to the Django Admin. These may be people such as managers, department heads, and the like. For those users, having your code separated into apps will create automatic separation in the Django admin. This will make it easier for those folks to find and focus on the specific app they need, instead of sorting through the entire project.
  
  
   Next: Django App Of The
   
    {{ variable_time_period }}
   
  
  
   Now that we've set a bit of foundation, we'll be looking at various Django apps and how they can be used. Some will be apps we're currently using on projects. Some will be apps we think are interesting. We probably won't be doing an app every week, but we'll try to keep the cadence regular enough to be compelling! Stay tuned!

What IS a Django App?

Step-by-step guide to integrating Meilisearch with Django, complete with automatic indexing, typo tolerance, and relevant filtering capabilities.

# compose.yml 
services:
 search:
 image: getmeili/meilisearch:v1.7
 volumes:
 - ./meili-data:/meili_data
 ports:
 - "7700:7700"

docker-compose up -d search

pip install meilisearch

# settings.py
SEARCH_API_URL = env("SEARCH_API_URL", default="http://search:7700/")
SEARCH_API_KEY = env("SEARCH_API_KEY", default=None)
SEARCH_INDEXES = {
 "main": "main_search",
 # You can define additional indexes here for different purposes
 # "autocomplete": "autocomplete_search",
 # "admin": "admin_search",
}
# Control whether we index the search or not
INDEX_SEARCH = env.bool("INDEX_SEARCH", default=True)

# products/models.py
from django.db import models

class Product(models.Model):
 name = models.CharField(max_length=255)
 description = models.TextField(blank=True)
 price = models.DecimalField(max_digits=10, decimal_places=2)
 sku = models.CharField(max_length=50, unique=True)
 brand = models.CharField(max_length=100, blank=True)
 active = models.BooleanField(default=True)
 created_at = models.DateTimeField(auto_now_add=True)
 updated_at = models.DateTimeField(auto_now=True)

 def get_absolute_url(self):
 return f"/products/{self.id}/"

 def __str__(self):
 return self.name

class Product(models.Model):
 name = models.CharField(max_length=255)
 sku = models.CharField(max_length=50, unique=True)
 description = models.TextField()
 price = models.DecimalField(max_digits=10, decimal_places=2)

class Category(models.Model):
 name = models.CharField(max_length=255)
 slug = models.SlugField(max_length=100)
 description = models.TextField()
 active = models.BooleanField(default=True)

# search/schemas.py
from pydantic import BaseModel

class SearchSchema(BaseModel):
 """
 Pydantic model for our main search schema
 """
 id: str
 title: str | None = None
 type: str # 'product', 'category', etc.
 url: str
 active: bool = True

 # Product fields
 product_name: str | None = None
 product_description: str | None = None
 brand_name: str | None = None
 sku: str | None = None

# products/models.py
from django.db import models
from search.schemas import SearchSchema

class Product(models.Model):
 # fields and methods as before

 def search_data(self):
 """Return data for the search index"""
 return SearchSchema(
 # prefix the ID with the "type" for the SearchSchema
 id=f"product-{self.id}",
 title=self.name,
 type="product",
 url=self.get_absolute_url(),
 active=self.active,
 product_name=self.name,
 product_description=self.description,
 brand_name=self.brand,
 sku=self.sku,
 )

def get_meilisearch_client():
 """Return a meilisearch client"""
 url = settings.SEARCH_API_URL
 api_key = settings.SEARCH_API_KEY
 client = meilisearch.Client(url, api_key)
 return client

def update_object(obj):
 """Update a single object in the search index or delete if inactive"""
 client = get_meilisearch_client()
 index_name = settings.SEARCH_INDEXES["main"]
 index = client.index(index_name)

 if obj["active"]:
 # Update the object in the index
 index.add_documents([obj])
 else:
 # Delete the object from the index
 index.delete_document(obj["id"])

def build_main_indexes(verbose=False):
 """Build the main search index with all searchable models"""
 from products.models import Product # Import your models

 client = get_meilisearch_client()
 index_name = settings.SEARCH_INDEXES["main"]
 index = client.index(index_name)

 # Configure the index settings
 index_settings = {
 "filterableAttributes": ["type", "active"],
 "searchableAttributes": [
 "title",
 "product_name",
 "product_description",
 "sku",
 "brand_name",
 ],
 }
 index.update_settings(index_settings)

 # Index all active products
 products = Product.objects.filter(active=True)
 if verbose:
 print(f"Indexing {products.count()} products...")

 product_docs = [product.search_data().dict() for product in products]
 if product_docs:
 index.add_documents(product_docs)

def rebuild_main_indexes(verbose=False):
 """Completely rebuild the main search index"""
 client = get_meilisearch_client()
 index_name = settings.SEARCH_INDEXES["main"]

 # Delete the index if it exists
 try:
 client.delete_index(index_name)
 except meilisearch.errors.MeiliSearchApiError:
 pass

 # Create the index and build it
 client.create_index(index_name)
 build_main_indexes(verbose=verbose)

# search/mixins.py
from django.conf import settings
from search.utils import update_object

class UpdateSearchMixin:
 """
 Mixin to update the search index when a model is saved
 """
 def update_search(self):
 if settings.INDEX_SEARCH:
 update_object(self.search_data().dict())

 def save(self, *args, **kwargs):
 """ Override the save method so we can call update_search after save """
 # Call the original save method
 super().save(*args, **kwargs)
 # Update the search index
 self.update_search()

# products/models.py
from django.db import models
from search.schemas import SearchSchema
from search.mixins import UpdateSearchMixin

# Add UpdateSearchMixin
class Product(UpdateSearchMixin, models.Model):
 ...

# search/management/commands/index_search.py
from __future__ import annotations

import djclick as click

from search.utils import rebuild_main_indexes


@click.command()
@click.option("--verbose", is_flag=True, default=False)
def main(verbose):
 """Index objects in Meilisearch."""
 click.secho("Rebuilding main indexes...", fg="green", nl=False)
 rebuild_main_indexes(verbose=verbose)
 click.echo("Indexes built.")

# search/management/commands/index_search.py
from __future__ import annotations

import time
import djclick as click
from django.conf import settings
from products.models import Product, Category
from search.utils import get_meilisearch_client

@click.command()
@click.option("--verbose", is_flag=True, default=False)
@click.option("--batch-size", default=1000, help="Batch size for indexing")
def main(verbose, batch_size):
 """Rebuild search indexes in Meilisearch with batch processing."""
 start_time = time.time()
 click.secho("Rebuilding search indexes...", fg="green")

 # Get client and recreate index
 client = get_meilisearch_client()
 index_name = settings.SEARCH_INDEXES["main"]

 # Delete and recreate index
 try:
 client.delete_index(index_name)
 except Exception:
 pass

 client.create_index(index_name)
 index = client.index(index_name)

 # Configure index settings
 click.echo("Configuring index settings...")
 index_settings = {
 "filterableAttributes": ["type", "active"],
 "searchableAttributes": [
 "title", "product_name", "product_description", 
 "sku", "brand_name", "category_name"
 ],
 }
 index.update_settings(index_settings)

 # Index products in batches
 products = Product.objects.filter(active=True)
 total_products = products.count()

 if verbose:
 click.echo(f"Indexing {total_products} products in batches of {batch_size}...")

 # Process in batches to avoid memory issues with large datasets
 batch_count = 0
 for i in range(0, total_products, batch_size):
 batch = products[i:i+batch_size]
 product_docs = [p.search_data().dict() for p in batch]
 if product_docs:
 index.add_documents(product_docs)

 batch_count += 1
 if verbose:
 click.echo(f"Processed batch {batch_count}, {min(i+batch_size, total_products)}/{total_products} products")

 # Index categories (typically smaller, so we do it all at once)
 categories = Category.objects.filter(active=True)
 category_docs = [cat.search_data().dict() for cat in categories]
 if category_docs:
 index.add_documents(category_docs)

 elapsed_time = time.time() - start_time
 click.secho(f"✓ Indexing completed in {elapsed_time:.2f} seconds", fg="green")

# products/models.py
class Category(UpdateSearchMixin, models.Model):
 name = models.CharField(max_length=255)
 description = models.TextField(blank=True)
 active = models.BooleanField(default=True)

 def get_absolute_url(self):
 return f"/categories/{self.id}/"

 def search_data(self):
 return SearchSchema(
 id=f"category-{self.id}",
 title=self.name,
 type="category",
 url=self.get_absolute_url(),
 active=self.active,
 category_name=self.name,
 category_description=self.description,
 products_count=self.products.count(),
 )

# search/schemas.py
class SearchSchema(BaseModel):
 """
 Pydantic model for our main search schema
 """
 # other fields as before... 

 # Category fields
 category_name: str | None = None
 category_description: str | None = None
 products_count: int | None = None

def build_main_indexes(verbose=False):
 """Build the main search index with all searchable models"""
 from products.models import Product, Category # Import your models

 client = get_meilisearch_client()
 index_name = settings.SEARCH_INDEXES["main"]
 index = client.index(index_name)

 # Configure the index settings
 index_settings = {
 "filterableAttributes": ["type", "active"],
 "searchableAttributes": [
 ...
 "category_name",
 "category_description",
 ],
 }
 index.update_settings(index_settings)

 # Index all active products as before... 

 # Index all active categories
 categories = Category.objects.filter(active=True)
 if verbose:
 print(f"Indexing {categories.count()} categories...")

 category_docs = [cat.search_data().dict() for cat in categories]
 if category_docs:
 index.add_documents(category_docs)

# search/views.py
from django.shortcuts import render
from django.conf import settings
from search.utils import get_meilisearch_client

def search(request):
 """Search view"""
 query = request.GET.get("q", "").strip()
 type_filter = request.GET.get("type", None)
 results = []

 if query:
 client = get_meilisearch_client()
 index = client.index(settings.SEARCH_INDEXES["main"])

 # Build filter string using your filterable attributes 
 filter_str = "active = true"
 if type_filter:
 filter_str += f" AND type = {type_filter}"

 # Perform search
 search_results = index.search(
 query, 
 {
 "filter": filter_str,
 "limit": 50
 }
 )
 results = search_results["hits"]

 return render(
 request,
 "search/results.html",
 {
 "results": results,
 "query": query,
 "type_filter": type_filter
 },
 )


{% extends "base.html" %}

{% block content %}
<div class="search-container">
 
 <form method="get" action="{% url 'search' %}">
 <input type="text" name="q" value="{{ query }}" placeholder="Search...">
 <button type="submit">Search</button>
 </form>

 
 {% if query %}
 <h2>Results for "{{ query }}"</h2>
 {% if results %}
 <div class="search-results">
 {% for result in results %}
 <div class="search-result">
 <h3><a href="{{ result.url }}">{{ result.title }}</a></h3>
 </div>
 {% endfor %}
 </div>
 {% else %}
 No results found.
 {% endif %}
 {% endif %}
</div>
{% endblock %}

# urls.py
from django.urls import path
from search.views import search

urlpatterns = [
 # ... other URLs
 path('search/', search, name='search'),
]

 
 
 For a production application, you might want to enhance this with:
 
 
 
 
 Pagination
 
 : For handling large result sets
 
 
 
 Highlighting
 
 : To show users where their search terms matched
 
 
 
 Faceted search
 
 : To allow filtering by multiple attributes
 
 
 
 Autocomplete
 
 : To suggest search terms as users type
 
 
 
 Analytics
 
 : To track popular searches and improve your search experience
 
 
 
 Meilisearch supports all these features through its API, making it easy to build a sophisticated search experience as your application grows.
 
 
 See It In Action: Surplus Sales Case Study
 
 
 Want to see what a production Meilisearch implementation looks like? Visit our client
 
 Surplus Sales
 
 and try out their search functionality. Pay special attention to:
 
 
 
 
 Typo tolerance
 
 : Try searching for "trnasistors" (misspelled) and notice how it still finds what you need
 
 
 
 Speed
 
 : The millisecond response times, even when filtering through their 70,000+ item catalog
 
 
 
 Relevance
 
 : How the most relevant parts appear at the top of results
 
 
 
 Complex metadata handling
 
 : Search by part numbers, specifications, or descriptions
 
 
 
 Their implementation handles all the concepts we've covered in this tutorial. It's a great example of how these techniques scale to real-world applications.
 
 
 Alternative Approach: Using django-meili
 
 
 If you prefer a more Django-native approach with less custom code, you might want to check out
 
 django-meili
 
 . This package provides a more Haystack-like experience for integrating Meilisearch with Django. I haven't used it, but from what I can tell, these are the main differences between using django-meili and implementing something more like our custom Meilisearch implementation:
 
 
 
 
 Model-focused
 
 : django-meili is designed around indexing individual Django models, which works well if you're searching just one model at a time
 
 
 
 Less flexible
 
 : Our unified schema approach gives you more control over how different models are represented in search results
 
 
 
 Easier setup
 
 : django-meili requires less boilerplate code to get started
 
 
 
 Django-native
 
 : It follows Django conventions more closely with a familiar API
 
 
 
 django-meili might be a good choice if:
 
 
 
 You're primarily searching within a single model type
 
 
 You prefer a more Django-integrated approach with less custom code
 
 
 You don't need the flexibility of a unified schema across different model types
 
 
 
 Further Reading
 
 
 
 
 Meilisearch Official Documentation
 
 
 
 
 Meilisearch Python Client
 
 
 
 
 Meilisearch Settings Reference
 
 
 
 
 Meilisearch Search Parameters
 
 
 
 
 django-meili GitHub Repository
 
 
 
 
 Need help implementing advanced search for your Django project?
 
 Contact our team at Revsys
 
 for expert
 
 Django consulting
 
 . We've implemented search solutions for clients ranging from small startups to large enterprises, and we'd be happy to help you build a search experience that delights your users.

How to Add Blazing Fast Search to Your Django Site with Meilisearch

Test output is often an afterthought. You can improve your flow and slightly gamify your bug hunting by combining the wonderful Rich Python library and pytest.

<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 <h2>
 Debugging complex code is not usually fun
 </h2>
 
 It's hard to get motivated when faced with a big pile of numbers that just aren't lining up in the ways they are supposed to. Stupid numbers, do what you're told!
 
 <h2>
 The task at hand
 </h2>
 
 I've been rebuilding a 15 to 20-year-old ColdFusion currency trading learning simulation for a client. The math involved is mostly contained in SQL Server stored procedures, some values in tables are really constants (or at least most of the time), and variable names between steps of the process aren't consistent. This has made the rebuilding effort harder than it originally appeared. Add in finding some undesired behaviors in the old app and the desire to improve these situations going forward has been a bit of a slog actually.
 
 
 I knew that today was the day I had to just dig in and iterate on this code until all of the numbers were correct now that I had a known good set of values to work against.
 
 
 Since I needed to compare several numbers together, I thought I might use a
 <a href="https://rich.readthedocs.io/en/stable/tables.html">
 Rich Table
 </a>
 to make the output a little better organized. And I'm VERY glad I did.
 
 
 Here is what I started with:
 
 </div>
</div>
<div class="block-content">
 
 
 <img alt="Initial boring pytest output" class="richtext-image full-width" height="742" src="https://sfo2.digitaloceanspaces.com/revsys-com-prod-media/images/CleanShot_2024-02-03_at_08.18.27.width-800.png" width="800"/>
 
 
</div>
<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 
 Not exactly pretty. It spit out the numbers it was calculating and I could compare them against the known good values visually. Looking and re-re-reading the numbers a half dozen times I was already bored and feeling tired despite having too much caffeine in my system.
 
 
 So I implemented all of the expected, calculated and differences between the two in a Rich Table. I took a couple of moments to adjust some colors and right justifying all of these numbers because I knew I was going to be staring at this
 
 all freaking day
 
 .
 
 </div>
</div>
<div class="block-content">
 
 
 <img alt="Initial Python Rich table formatting" class="richtext-image full-width" height="475" src="https://sfo2.digitaloceanspaces.com/revsys-com-prod-media/images/CleanShot_2024-02-03_at_09.15.54.width-800.png" width="800"/>
 
 
</div>
<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 <h2>
 The debugging
 </h2>
 
 I then went about debugging the first
 
 Total Unhedged
 
 number because all of the following calculations depended on this number being correct. And I quickly found the bug!
 
 
 I would love to say it was because of this great new visualization, but it was just closely re-reading my Python code and comparing it (
 
 yet again
 
 ) to the original portion of a stored procedure to find my subtle mistake.
 
 
 One down, 6 more to go I thought. This is going to be a long day, but it turned out two more of the values were
 
 NEARLY
 
 correct now, just off due to rounding. This
 
 WAS
 
 easier to spot because of the tabular layout. I treated myself to a bit more table improvement, I wrote a little function named
 <code>
 calc_diff()
 </code>
 that took the expected and actual values and color coded the difference red if there was a difference and green if not.
 
 </div>
</div>
<div class="block-content">
 
 
 <img alt="Improved table with red/green color coding" class="richtext-image full-width" height="341" src="https://sfo2.digitaloceanspaces.com/revsys-com-prod-media/images/CleanShot_2024-02-03_at_09.36.37.width-800.png" width="800"/>
 
 
</div>
<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 <h2>
 Table code example
 </h2>
 
 The code for this is surprisingly easy.
 <a href="https://github.com/Textualize/rich">
 Rich
 </a>
 was already a dependency of something else in my project's virtualenv so I didn't even need to install it. Here is a simplified example of the code for you:
 
 </div>
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>from decimal import Decimal

from rich.console import Console
from rich.table import Table
from rich.text import Text 

def calc_diff(expected, actual):
 diff = expected - actual

 color = "red"
 if diff == 0:
 color = "green"

 text = Text(str(diff))
 text.stylize(color)

 return text

# My actual pytest test
def test_ex3_math(ex3_params):
 expected_total_unhedged = Decimal("3.394148")

 # call a method on a fixture class that does the math calculation for us, yours
 # would obviously be different 
 total_unhedged = ex3_params.total_unhedged()

 # Build our table
 table = Table(
 title="Exercise 3 Answers", title_style="gold1", style="deep_sky_blue3"
 )
 table.add_column(
 "Name", style="turquoise2", header_style="turquoise2", no_wrap=True
 )
 table.add_column("Expected", header_style="turquoise2", justify="right")
 table.add_column("Actual", header_style="turquoise2", justify="right")
 table.add_column("Difference", header_style="turquoise2", justify="right")

 # Total Unhedged - just showing you one row here for sake of brevity
 table.add_row(
 "Total Unhedged",
 str(expected_total_unhedged),
 str(total_unhedged),
 calc_diff(expected_total_unhedged, total_unhedged),
 )

 # Output the table 
 console = Console(color_system="256")
 console.print(table)

 # Trigger our test to fail artificially 
 assert False
</pre>
 </div>
</div>
<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 <h2>
 Conclusion
 </h2>
 
 Was it worth the extra time?
 
 DEFINITELY!
 
 I not only made it easier to compare the values as I worked but also motivated me a little more than usual as I turned each red line green.
 
 
 Every little bit helps, right? Hopefully, you enjoyed this protip and can put it use in one of your upcoming debugging sessions.
 
 
 If you or your company struggles with testing your Python or Django apps, check out
 <a href="/teststart/">
 TestStart
 </a>
 our testing consulting package to help jump start your team into better productivity.
 
 </div>
</div>

from decimal import Decimal

from rich.console import Console
from rich.table import Table
from rich.text import Text 

def calc_diff(expected, actual):
        diff = expected - actual

        color = "red"
        if diff == 0:
            color = "green"

        text = Text(str(diff))
        text.stylize(color)

        return text

# My actual pytest test
def test_ex3_math(ex3_params):
    expected_total_unhedged = Decimal("3.394148")

    # call a method on a fixture class that does the math calculation for us, yours
   # would obviously be different 
    total_unhedged = ex3_params.total_unhedged()

    # Build our table
    table = Table(
        title="Exercise 3 Answers", title_style="gold1", style="deep_sky_blue3"
    )
    table.add_column(
        "Name", style="turquoise2", header_style="turquoise2", no_wrap=True
    )
    table.add_column("Expected", header_style="turquoise2", justify="right")
    table.add_column("Actual", header_style="turquoise2", justify="right")
    table.add_column("Difference", header_style="turquoise2", justify="right")

    # Total Unhedged - just showing you one row here for sake of brevity
    table.add_row(
        "Total Unhedged",
        str(expected_total_unhedged),
        str(total_unhedged),
        calc_diff(expected_total_unhedged, total_unhedged),
    )

   # Output the table 
   console = Console(color_system="256")
   console.print(table)

   # Trigger our test to fail artificially 
   assert False

 


 
  
   Conclusion
  
  
   Was it worth the extra time?
   
    DEFINITELY!
   
   I not only made it easier to compare the values as I worked but also motivated me a little more than usual as I turned each red line green.
  
  
   Every little bit helps, right? Hopefully, you enjoyed this protip and can put it use in one of your upcoming debugging sessions.
  
  
   If you or your company struggles with testing your Python or Django apps, check out
   
    TestStart
   
   our testing consulting package to help jump start your team into better productivity.

"En-Rich" your Python testing

A simple import trick to make it easy to keep your django-ninja view functions in different Django apps and files.

<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 
 I've been evaluating
 <a href="https://django-ninja.rest-framework.com/">
 django-ninja
 </a>
 recently, using it in a personal project to see when it might be appropriate for
our clients.
 
 <h2>
 The Problem - How to have multiple view files
 </h2>
 
 One thing that was not clear was how to break up your API views and sprinkle
them around in multiple files. The default tutorial and documentation is not
bad, but just does not cover this small thing.
 
 
 The way django-ninja works is that you decorate functional views with
 <code>
 @api
 </code>
 where you indicate what HTTP method it is for and some other parameters.
Everything is perfect until you realize your views.py has grown huge or you
want to include api views from several Django applications.
 
 
 For those of us who are very familiar with DRF, it is not immediately clear how
you can break your views into multiple files.
 
 
 Let's use an example of having a Django apps named
 <code>
 core
 </code>
 ,
 <code>
 users
 </code>
 , and
 <code>
 posts
 </code>
 to mimic some sort of blog like software and our project directory, which contains
our settings and urls files will be named
 <code>
 config/
 </code>
 which is the standard way
we roll at REVSYS.
 
 
 Your first guess might be that you should create another instance of
 <code>
 NinjaAPI
 </code>
 and use that to decorate your views in each file. Unfortunately, because of how
the internal registry works for these, that would create multiple django-ninja
"sites", each with their own docs, etc. which is obviously not what we want.
 
 <h2>
 The Solution
 </h2>
 
 The easiest way to do this is to pass around your API decorator and ensure
your view files are imported before you call on Ninja to generate your URLs.
 
 
 While I hate having an app named
 <code>
 core
 </code>
 , I used it in this case. So we
create our API instance with the following:
 
 <div class="codehilite">
 <pre><code># File: core/api.py
from ninja import NinjaAPI

api = NinjaAPI(csrf=True)
</code></pre>
 </div>
 
 And then in our views we import this and use it:
 
 <div class="codehilite">
 <pre><code># File: users/views.py
from core.api import api

@api.get("users/something/"):
def get_user(request):
 # ... do something interesting here ...

# File: posts/views.py
from core.api import api

@api.get("posts/{username}/"):
def get_posts(requests, username: str):
 # ... do something interesting here ...
</code></pre>
 </div>
 
 Then the final bit is to make sure we import these files in our
 <code>
 urls.py
 </code>
 before
we call
 <code>
 api.urls
 </code>
 like this:
 
 <div class="codehilite">
 <pre><code># File: config/urls.py
import users.views
import posts.views
from core.api import api

urlpatterns = [
 path("api/", api.urls),
]
</code></pre>
 </div>
 <h2>
 Alternative Solution
 </h2>
 
 Turns out there is also a DRF like concept of
 <a href="https://django-ninja.rest-framework.com/guides/routers/">
 routers
 </a>
 which is honestly probably a better more
maintainable solution than the above.
 
 
 Thanks to
 <a href="https://twitter.com/uninen/status/1622007355067056132?s=46&amp;t=6Jl-2SgFBle7KLjsma1uQw">
 Ville S&auml;&auml;vuori
 </a>
 for point this out after I posted this.
 
 
 Hopefully, this saves you a bit of time when you first start using django-ninja!
 
 
 If you need help creating Django REST APIs don't hesitate to
 <a href="/contact/">
 reach out to us
 </a>
 !
 
 </div>
</div>

# File: core/api.py
from ninja import NinjaAPI

api = NinjaAPI(csrf=True)

# File: users/views.py
from core.api import api

@api.get("users/something/"):
def get_user(request):
   # ... do something interesting here ...

# File: posts/views.py
from core.api import api

@api.get("posts/{username}/"):
def get_posts(requests, username: str):
   # ... do something interesting here ...

# File: config/urls.py
import users.views
import posts.views
from core.api import api

urlpatterns = [
    path("api/", api.urls),
]

  
  
   Alternative Solution
  
  
   Turns out there is also a DRF like concept of
   
    routers
   
   which is honestly probably a better more
maintainable solution than the above.
  
  
   Thanks to
   
    Ville Säävuori
   
   for point this out after I posted this.
  
  
   Hopefully, this saves you a bit of time when you first start using django-ninja!
  
  
   If you need help creating Django REST APIs don't hesitate to
   
    reach out to us
   
   !

Breaking up django-ninja views

Sometimes you want your FloatFields to be nullable, but your JS code isn&#x27;t quite smart enough to handle it.

<div class="block-content">
 
 I run into this situation every couple of years, and I always forget the solution. Even asked around the office and everyone had the same "Oh yeah, I remember doing that once... but now how".
 
 
 When these situations happen to us, it is often time to write a blog post!
 
 
 The situation I was faced with was I had:
 
 <ul>
 <li data-block-key="7lj4t">
 A model with a few nullable Django Model FloatFields
 </li>
 <li data-block-key="b2ksr">
 Django Rest Framework, using a ModelSerializer
 </li>
 <li data-block-key="rfo1">
 react-hook-form, react-query and standard fetch calls to POST the data to the endpoint
 </li>
 </ul>
 
 The issue is that we have a simple HTML text input to collect a possible float value. It's easy enough to mark this as not required on the form and any client-side form validation. However, it is NOT always that easy to handle what happens next. Because it's a text field, the JS tooling wants to send it as an empty string. So we end up with a payload that looks like:
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>{
 "name": "Frank Wiles",
 "weight": "",
}
</pre>
 </div>
</div>
<div class="block-content">
 
 When what we'd really like to have is:
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>{
 "name": "Frank Wiles",
 "weight": null,
}
</pre>
 </div>
</div>
<div class="block-content">
 
 DRF rejects this saying that an empty string is not a number. It is correct of course, just not being super helpful in this exact situation. So what is the solution? Enter BlankableFloatField
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>from rest_framework.serializers import FloatField


class BlankableFloatField(FloatField):
 """
 This accepts an empty string as None
 """

 def to_internal_value(self, data):
 if data == "":
 return None
 return super().to_internal_value(data)
</pre>
 </div>
</div>
<div class="block-content">
 
 You can override this on your ModelSerializer by just specifying the fields manually like this:
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>class PersonSerializer(serializers.ModelSerializer):
 # ... your other fields here ... 
 weight = BlankableFloatField(required=False, allow_null=True, default=None)
</pre>
 </div>
</div>
<div class="block-content">
 
 The solution isn't hard to execute, but it's hard to reason about and figure out the
 
 best
 
 place to adjust this in your Django project.
 
 
 Hopefully, the next time you find yourself with Javascript error in your console saying that an empty string is not a number this helps you out!
 
</div>

{
  "name": "Frank Wiles",
  "weight": "",
}

 
{
  "name": "Frank Wiles",
  "weight": null,
}

 
from rest_framework.serializers import FloatField


class BlankableFloatField(FloatField):
    """
    This accepts an empty string as None
    """

    def to_internal_value(self, data):
        if data == "":
            return None
        return super().to_internal_value(data)

 
class PersonSerializer(serializers.ModelSerializer):
    # ... your other fields here ... 
    weight = BlankableFloatField(required=False, allow_null=True, default=None)

 


 
  The solution isn't hard to execute, but it's hard to reason about and figure out the
  
   best
  
  place to adjust this in your Django project.
 
 
  Hopefully, the next time you find yourself with Javascript error in your console saying that an empty string is not a number this helps you out!

Sometimes you want your FloatFields to be nullable, but your JS code isn't quite smart enough to handle it.

Nullable DRF FloatFields and Javascript

I needed to structure permissions in an API view that had multiple user roles, and I learned about using the &amp; (and), | (or) and ~ (not) operators with Django REST Framework permissions.

<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 
 Recently, I needed to structure permissions in an API view so that someone could access an API if they were a superuser or they had some other status, like being a manager. The
 <a href="https://www.django-rest-framework.org/api-guide/permissions/#how-permissions-are-determined">
 Django Rest Framework Permissions docs
 </a>
 say that &ldquo;If any permission check fails an exceptions.PermissionDenied or exceptions.NotAuthenticated exception will be raised,&rdquo; which made me think I&rsquo;d have to override the
 <a href="http://www.cdrf.co/3.9/rest_framework.views/APIView.html">
 <code>
 get_permissions()
 </code>
 </a>
 method of the view or write a series of complicated
 <code>
 IsUserTypeA
 </code>
 ,
 <code>
 IsUserTypeAOrUserTypeB
 </code>
 permissions classes, in order to do what I wanted. That didn&rsquo;t seem ideal.
 
 
 Then a
 <a href="https://twitter.com/webology">
 colleague
 </a>
 pointed out one line in the DRF docs further down on the
 <a href="https://www.django-rest-framework.org/api-guide/permissions/#setting-the-permission-policy">
 same page
 </a>
 : &ldquo;Note: it supports &amp; (and), | (or) and ~ (not).&rdquo; This was exactly what I needed.
 
 
 Here&rsquo;s how it works.
 
 <h2>
 All permissions must return True
 </h2>
 
 The expected way that DRF permissions work is as stated above; when you list multiple permission classes in
 <code>
 permission_classes
 </code>
 , each of them must return True in order for the user to have access to that view. This is most likely what you&rsquo;re used to when implementing DRF permissions.
 
 <div class="codehilite">
 <pre><code>from rest_framework.permissions import IsAuthenticated, IsAdminUser, ReadOnly
from rest_framework.viewsets import ModelViewSet

from my_app.permissions import IsStudent, IsFaculty


class MyModelViewSet(ModelViewSet):
 permission_classes = (IsAuthenticated, IsAdminUser)
</code></pre>
 </div>
 
 In this case, the user must be authenticated
 
 and
 
 be an admin user, or else they cannot access any part of the view.
 
 <h2>
 The | (or) operator
 </h2>
 
 Using the
 <code>
 |
 </code>
 (or) operator in DRF permissions allows you to set multiple conditions the user might meet and be able to access this view. For instance, for an educational app, a view might be available to students
 
 or
 
 faculty.
 
 <div class="codehilite">
 <pre><code>from rest_framework.permissions import IsAuthenticated, IsAdminUser, ReadOnly
from rest_framework.viewsets import ModelViewSet

from my_app.permissions import IsStudent, IsFaculty


class MyModelViewSet(ModelViewSet):
 permission_classes = (IsAdminUser | IsStudent | IsFaculty)
</code></pre>
 </div>
 
 This expression
 <code>
 (IsAdminUser | IsStudent | IsFaculty)
 </code>
 allows a user who is an admin, a student, or a faculty member to access the view. Instead of needing to pass all three of these conditions, a user only needs to pass one.
 
 
 The way I accomplished this before was a custom permissions class named something like
 <code>
 IsAdminStudentOrFaculty
 </code>
 , but with multiple roles and complex permissions, this meant creating custom permission classes for nearly every combination of user role. (In this example, I refer to custom permissions
 <code>
 IsStudent
 </code>
 and
 <code>
 IsFaculty
 </code>
 ; there are
 <a href="https://www.django-rest-framework.org/api-guide/permissions/#examples">
 some examples in the docs
 </a>
 of how to implement custom permissions like these.)
 
 <h2>
 The &amp; (and) operator
 </h2>
 
 Especially combined with the
 <code>
 |
 </code>
 (or) operator, the
 <code>
 &amp;
 </code>
 (and) operator is really powerful. It enables you to set clear permissions boundaries without writing increasingly complex custom permissions classes.
 
 <div class="codehilite">
 <pre><code>from rest_framework.permissions import IsAuthenticated, IsAdminUser, ReadOnly
from rest_framework.viewsets import ModelViewSet

from my_app.permissions import IsStudent, IsFaculty


class MyModelViewSet(ModelViewSet):
 permission_classes = (IsAuthenticated &amp; (IsAdminUser | IsFaculty | ReadOnly))
</code></pre>
 </div>
 
 This example,
 <code>
 (IsAuthenticated &amp; (IsAdminUser | IsFaculty | ReadOnly))
 </code>
 , requires that all users accessing this view be authenticated. In addition, it sets other permissions: a user must be authenticated
 
 and
 
 be an admin or a faculty member, or they only have &ldquo;read&rdquo; access to this view.
 
 <h2>
 The
 <code>
 ~
 </code>
 (not) operator
 </h2>
 
 The
 <code>
 ~
 </code>
 (not) operator allows you to simplify your permissions to permit everyone
 
 except
 
 users who meet specific criteria. Personally, I prefer to set permissions with positive language, like &ldquo;allow admins and faculty&rdquo; rather than &ldquo;don&rsquo;t allow students,&rdquo; but there are probably cases where the
 <code>
 ~
 </code>
 operator is a great option.
 
 <div class="codehilite">
 <pre><code>from rest_framework.permissions import IsAuthenticated, IsAdminUser, ReadOnly
from rest_framework.viewsets import ModelViewSet

from my_app.permissions import IsStudent, IsFaculty


class MyModelViewSet(ModelViewSet):
 permission_classes = (~IsStudent &amp; IsAuthenticated)
</code></pre>
 </div>
 
 <code>
 (~IsStudent &amp; IsAuthenticated)
 </code>
 says that anyone who is authenticated and who is
 
 not
 
 a student can access this view.
 
 
 I hope you, like me, now can free yourself from the cumbersome task of writing too many custom permission classes in your apps with several different types of user roles and instead make use of simpler permission classes and the
 <code>
 &amp;
 </code>
 ,
 <code>
 |
 </code>
 and
 <code>
 ~
 </code>
 operators!
 
 <h2>
 Further Reading
 </h2>
 <ul>
 <li>
 <a href="https://www.django-rest-framework.org/api-guide/permissions">
 Django REST Framework Permissions
 </a>
 docs
 </li>
 <li>
 The
 <a href="https://github.com/encode/django-rest-framework/pull/5753">
 pull request
 </a>
 that added the
 <code>
 &amp;
 </code>
 (and) and
 <code>
 |
 </code>
 (or) operators. This, and the next pull request, are also the inspiration for some of my examples.
 </li>
 <li>
 The
 <a href="https://github.com/encode/django-rest-framework/pull/6361">
 pull request
 </a>
 that added the
 <code>
 ~
 </code>
 (not) operator
 </li>
 </ul>
 
 
 Thanks to
 <a href="https://twitter.com/webology">
 Jeff Triplett
 </a>
 for reviewing a draft of this article and finding this tip in the docs in the first place!
 
 
 </div>
</div>

from rest_framework.permissions import IsAuthenticated, IsAdminUser, ReadOnly
from rest_framework.viewsets import ModelViewSet

from my_app.permissions import IsStudent, IsFaculty


class MyModelViewSet(ModelViewSet):
    permission_classes = (IsAuthenticated, IsAdminUser)

from rest_framework.permissions import IsAuthenticated, IsAdminUser, ReadOnly
from rest_framework.viewsets import ModelViewSet

from my_app.permissions import IsStudent, IsFaculty


class MyModelViewSet(ModelViewSet):
    permission_classes = (IsAdminUser | IsStudent | IsFaculty)

from rest_framework.permissions import IsAuthenticated, IsAdminUser, ReadOnly
from rest_framework.viewsets import ModelViewSet

from my_app.permissions import IsStudent, IsFaculty


class MyModelViewSet(ModelViewSet):
    permission_classes = (IsAuthenticated & (IsAdminUser | IsFaculty | ReadOnly))

from rest_framework.permissions import IsAuthenticated, IsAdminUser, ReadOnly
from rest_framework.viewsets import ModelViewSet

from my_app.permissions import IsStudent, IsFaculty


class MyModelViewSet(ModelViewSet):
    permission_classes = (~IsStudent & IsAuthenticated)

  
  
   
    (~IsStudent & IsAuthenticated)
   
   says that anyone who is authenticated and who is
   
    not
   
   a student can access this view.
  
  
   I hope you, like me, now can free yourself from the cumbersome task of writing too many custom permission classes in your apps with several different types of user roles and instead make use of simpler permission classes and the
   
    &
   
   ,
   
    |
   
   and
   
    ~
   
   operators!
  
  
   Further Reading
  
  
   
    
     Django REST Framework Permissions
    
    docs
   
   
    The
    
     pull request
    
    that added the
    
     &
    
    (and) and
    
     |
    
    (or) operators. This, and the next pull request, are also the inspiration for some of my examples.
   
   
    The
    
     pull request
    
    that added the
    
     ~
    
    (not) operator
   
  
  
   
    Thanks to
    
     Jeff Triplett
    
    for reviewing a draft of this article and finding this tip in the docs in the first place!

I needed to structure permissions in an API view that had multiple user roles, and I learned about using the & (and), | (or) and ~ (not) operators with Django REST Framework permissions.

A Tip About DRF Permissions

I was working on a project with a coworker recently and I noticed in one of their pull requests that they used a custom exception in one of our Django REST Framework viewsets. I prefer this way to what I was doing before, so I wanted to share it with you!

<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 
 I was working on a project with a
 <a href="https://twitter.com/fwiles">
 coworker
 </a>
 recently and I noticed in one of their pull requests that they used a custom exception in one of our Django REST Framework viewsets. I prefer this way to what I was doing before, so I wanted to share it with you!
 
 <h2>
 My Less Clean, Less Consistent Exception Handling
 </h2>
 
 I made good use of the
 <a href="https://www.django-rest-framework.org/api-guide/status-codes/">
 status codes
 </a>
 page of the DRF docs. For example, if a user uploaded a CSV file that my view couldn&rsquo;t read, I might have had something like this:
 
 <div class="codehilite">
 <pre><code>from rest_framework import status
from rest_framework.response import Response
from rest_framework.views import APIView


class MyCSVUploadView(APIView):
 def post(self, request, *args, **kwargs):
 &hellip;
 if file_error:
 return Response(
 {"detail": "Unable to read file."}, 
 status=status.HTTP_400_BAD_REQUEST
 )
</code></pre>
 </div>
 
 This works just fine. But in some cases, I might return the same type of error response in multiple places. For a university that is no longer accepting applications, I might have multiple places where I am essentially passing the same &ldquo;Deadline passed&rdquo; message. Although the response is just one line of code, I have to find it, copy it, and paste it to make sure I&rsquo;m using the same wording, or deal with inconsistency in my exception messages.
 
 <h2>
 Custom Exception Classes for Clean Consistency
 </h2>
 
 For exception conditions that you encounter frequently in your code, you can override DRF&rsquo;s
 <code>
 APIException
 </code>
 class and create your own exceptions! Read more in
 <a href="https://www.django-rest-framework.org/api-guide/exceptions/#apiexception">
 the docs
 </a>
 .
 
 
 Import the
 <code>
 APIException
 </code>
 class and create a new class that inherits from it. Then set the
 <code>
 status_code
 </code>
 ,
 <code>
 default_detail
 </code>
 , and
 <code>
 default_code
 </code>
 attributes.
 
 <div class="codehilite">
 <pre><code># exceptions.py
from rest_framework.exceptions import APIException


class UnreadableCSVFile(APIException):
 status_code = 400
 default_detail = "Unable to read file."
 default_code = "unreadable_csv_file"
</code></pre>
 </div>
 
 The
 <code>
 status_code
 </code>
 attribute defines the HTTP status code you want to send in the response with this exception; refer to the
 <a href="https://www.django-rest-framework.org/api-guide/status-codes/">
 DRF list of status codes
 </a>
 for help. The
 <code>
 default_detail
 </code>
 attribute is the human-readable exception message you want to pass in the response. The
 <code>
 default_code
 </code>
 is a string that represents this specific exception.
 
 
 In your view, instead of manually returning a
 <code>
 Response()
 </code>
 object, raise your custom exception.
 
 <div class="codehilite">
 <pre><code># views.py
from rest_framework.views import APIView

From my_app.exceptions import UnreadableCSVFile

class MyCSVUploadView(APIView):
 def post(self, request, *args, **kwargs):
 &hellip;
 if file_error:
 raise UnreadableCSVFile()
</code></pre>
 </div>
 
 This allows my exceptions to be reusable across my app, which means the language I use in exception messages is always consistent.
 
 <h2>
 Custom Exception Handlers
 </h2>
 
 You can further customize your exceptions by creating a custom handler function (
 <a href="https://www.django-rest-framework.org/api-guide/exceptions/#custom-exception-handling">
 docs on custom exception handling
 </a>
 ). This is especially helpful if there are pieces of data, like the status code, that you want to ensure are included in all of your HTTP responses. This is a slightly different goal than what I refer to above, but it&rsquo;s worth knowing about. Here is the example from the docs:
 
 <div class="codehilite">
 <pre><code>from rest_framework.views import exception_handler

def custom_exception_handler(exc, context):
 # Call REST framework's default exception handler first, 
 # to get the standard error response.
 response = exception_handler(exc, context)

 # Now add the HTTP status code to the response.
 if response is not None:
 response.data['status_code'] = response.status_code

 return response
</code></pre>
 </div>
 
 This adds the
 <code>
 status_code
 </code>
 to each exception response sent in your app.
 
 
 You would then add your custom exception handler to the
 <code>
 REST_FRAMEWORK
 </code>
 settings in your
 <code>
 settings.py
 </code>
 so this custom exception handler will be executed every time there is an API exception:
 
 <div class="codehilite">
 <pre><code>REST_FRAMEWORK = {
 'EXCEPTION_HANDLER': 'my_project.my_app.utils.custom_exception_handler'
}
</code></pre>
 </div>
 
 For my example above, with the unreadable CSV file, the response would look like this:
 
 <div class="codehilite">
 <pre><code>{"status_code": 400, "detail": "Unable to read file."}
</code></pre>
 </div>
 
 With the status code returned as part of the response dictionary.
 
 
 
 Thanks to
 <a href="https://twitter.com/fwiles">
 Frank Wiles
 </a>
 for teaching me about custom exceptions, and
 <a href="https://twitter.com/webology">
 Jeff Triplett
 </a>
 for reviewing a draft of this article.
 
 
 </div>
</div>

from rest_framework import status
from rest_framework.response import Response
from rest_framework.views import APIView


class MyCSVUploadView(APIView):
    def post(self, request, *args, **kwargs):
        …
        if file_error:
            return Response(
                {"detail": "Unable to read file."}, 
                status=status.HTTP_400_BAD_REQUEST
            )

# exceptions.py
from rest_framework.exceptions import APIException


class UnreadableCSVFile(APIException):
    status_code = 400
    default_detail = "Unable to read file."
    default_code = "unreadable_csv_file"

# views.py
from rest_framework.views import APIView

From my_app.exceptions import UnreadableCSVFile

class MyCSVUploadView(APIView):
    def post(self, request, *args, **kwargs):
        …
        if file_error:
            raise UnreadableCSVFile()

from rest_framework.views import exception_handler

def custom_exception_handler(exc, context):
    # Call REST framework's default exception handler first, 
    # to get the standard error response.
    response = exception_handler(exc, context)

    # Now add the HTTP status code to the response.
    if response is not None:
        response.data['status_code'] = response.status_code

    return response

REST_FRAMEWORK = {
    'EXCEPTION_HANDLER': 'my_project.my_app.utils.custom_exception_handler'
}

{"status_code": 400, "detail": "Unable to read file."}

  
  
   With the status code returned as part of the response dictionary.
  
  
   
    Thanks to
    
     Frank Wiles
    
    for teaching me about custom exceptions, and
    
     Jeff Triplett
    
    for reviewing a draft of this article.

Custom Exceptions in Django REST Framework

When working with Wagtail, you might find that you&#x27;re using Wagtail Page models for some of your database models, but regular Django models for others. In this post, learn how to add your Django models to the Wagtail admin to make managing multiple types of models easier.

<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 <h2>
 Versions
 </h2>
 <ul>
 <li>
 Python 3.7
 </li>
 <li>
 Django 2.2
 </li>
 <li>
 Wagtail 2.6
 </li>
 </ul>
 
 When working with Wagtail, you might find that you're using Wagtail Page models for some of your database models, but regular Django models for others.
 
 
 A built-in example of this is the Django
 <code>
 User
 </code>
 model. When you log into the Wagtail admin, you can see the Django
 <code>
 User
 </code>
 model in the
 <code>
 Settings
 </code>
 submenu. The
 <code>
 User
 </code>
 model is not a Wagtail model; it's the same
 <code>
 User
 </code>
 model you see in a Django project that doesn't use Wagtail. Wagtail just exposes it to the Admin for you.
 
 </div>
</div>
<div class="block-content">
 
 
 <img alt="users_admin.png" class="richtext-image full-width" height="360" src="https://sfo2.digitaloceanspaces.com/revsys-com-prod-media/images/users_admin.width-800.png" width="404"/>
 
 
</div>
<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 
 We can do the same thing with our Django models: we can expose them to the Wagtail admin so we don't have to maintain two separate admin interfaces to manage our website content.
 
 
 For this example, let's assume we're working with these models:
 
 <div class="codehilite">
 <pre><code>from django.db import models 


class Pizza(models.Model):

 name = models.CharField(max_length=30)
 toppings = models.ManyToManyField("Topping")


class Topping(models.Model):

 name = models.CharField(max_length=30)
</code></pre>
 </div>
 <h2>
 Adding a single model
 </h2>
 
 The
 <a href="http://docs.wagtail.io/en/v2.6.1/reference/contrib/modeladmin/">
 Wagtail docs
 </a>
 are pretty clear on how to accomplish this, but let's walk through the steps.
 
 
 First, make sure
 <code>
 wagtail.contrib.modeladmin
 </code>
 is in your
 <code>
 INSTALLED_APPS
 </code>
 :
 
 <div class="codehilite">
 <pre><code># settings.py 

INSTALLED_APPS = [
 ...
 "wagtail.contrib.modeladmin",
]
</code></pre>
 </div>
 
 Next, in the same app as the model you want to expose to the Wagtail admin, add a file called
 <code>
 wagtail_hooks.py
 </code>
 .
 
 <div class="codehilite">
 <pre><code># wagtail_hooks.py

from wagtail.contrib.modeladmin.options import ModelAdmin, modeladmin_register 

from .models import Pizza


class PizzaAdmin(ModelAdmin):
 model = Pizza 
 menu_label = "Pizza" 
 menu_icon = "pick" 
 menu_order = 200 
 add_to_settings_menu = False 
 exclude_from_explorer = False 
 list_display = ("name",)
 list_filter = ("toppings",)
 search_fields = ("name",)


modeladmin_register(PizzaAdmin)
</code></pre>
 </div>
 
 Let's step through these options in the
 <code>
 ModelAdmin
 </code>
 class:
 
 <ul>
 <li>
 <code>
 model
 </code>
 : The name of the model you're adding.
 </li>
 <li>
 <code>
 menu_label
 </code>
 : Leave this blank to use the
 <code>
 verbose_name_plural
 </code>
 from your model. Give it a value to specify a new label for the Wagtail menu.
 </li>
 <li>
 <code>
 menu_icon
 </code>
 : Every menu item in the Wagtail admin has an icon, and you can specify the one you want to use. Here is a
 <a href="https://thegrouchy.dev/general/2015/12/06/wagtail-streamfield-icons.html">
 list of the available icons
 </a>
 .
 </li>
 <li>
 <code>
 menu_order
 </code>
 : What order you want this model to appear in. 000 is first, 100 is second, etc. Note: if you add multiple models to the admin, you won't get an error if two of them have the same
 <code>
 menu_order
 </code>
 ; Wagtail will just pick for you.
 </li>
 <li>
 <code>
 add_to_settings_menu
 </code>
 : Whether you want this menu item to appear in the
 
 Settings
 
 submenu in the Wagtail admin.
 </li>
 <li>
 <code>
 exclude_from_explorer
 </code>
 : Set to True if you
 
 do not
 
 want the explorer (the search box in the admin) to return results from this model. Set to False if you
 
 do
 
 want the explorer to return results from this model. (It's confusing.)
 </li>
 <li>
 <code>
 list_display
 </code>
 : Same as the Django admin; list the fields you want to display on the listing page for this model in the Wagtail admin.
 </li>
 <li>
 <code>
 list_filter
 </code>
 : Same as the Django admin; supply the fields you want to use to filter in the sidebar of the Wagtail admin.
 </li>
 <li>
 <code>
 search_fields
 </code>
 : Same as the Django admin; supply the fields that you want the explorer to use to return search results.
 </li>
 </ul>
 
 The final step is to register the admin class. Once you've done that and started your server, you'll be able to see your model in the Wagtail admin:
 
 </div>
</div>
<div class="block-content">
 
 
 <img alt="pizzas_admin.png" class="richtext-image full-width" height="328" src="https://sfo2.digitaloceanspaces.com/revsys-com-prod-media/images/pizzas_admin.width-800.png" width="800"/>
 
 
</div>
<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 <h2>
 Adding related models
 </h2>
 
 In our example models, we have two models:
 <code>
 Pizza
 </code>
 and
 <code>
 Toppings
 </code>
 . We could manually add the
 <code>
 Topping
 </code>
 model to the Wagtail admin and have it appear just below the
 <code>
 Pizza
 </code>
 model. We just learned how!
 
 
 But it's so closely related to the
 <code>
 Pizza
 </code>
 model that it might be nice if we were able to relate those two models together in a submenu, kind of like how
 
 Settings
 
 is its own submenu in the admin that contains Users, Redirects, Sites, etc.
 
 
 Go back to
 <code>
 wagtail_hooks.py
 </code>
 :
 
 <div class="codehilite">
 <pre><code># wagtail_hooks.py

from wagtail.contrib.modeladmin.options import (
 ModelAdmin, 
 ModelAdminGroup, 
 modeladmin_register 
)

from .models import Pizza, Topping


class PizzaAdmin(ModelAdmin):
 ...
 menu_order = 000 
 ...


class ToppingAdmin(ModelAdmin):
 model = Topping 
 menu_label = "Toppings" 
 menu_icon = "edit" 
 menu_order = 100 
 add_to_settings_menu = False 
 exclude_from_explorer = False 
 list_display = ("name",)
 search_fields = ("name",)
</code></pre>
 </div>
 
 Relating our two models together starts off in the same way: we create a class that inherits from
 <code>
 ModelAdmin
 </code>
 for each model and identify the necessary attributes like
 <code>
 model
 </code>
 and
 <code>
 menu_icon
 </code>
 to control things like their listing pages and search behavior.
 
 
 Then, we add a new class that inherits from
 <code>
 ModelAdminGroup
 </code>
 :
 
 <div class="codehilite">
 <pre><code># wagtail_hooks.py

from wagtail.contrib.modeladmin.options import (
 ModelAdmin, 
 ModelAdminGroup, 
 modeladmin_register 
)

from .models import Pizza, Topping


class PizzaAdmin(ModelAdmin):
 ...
 menu_order = 000 
 ...


class ToppingAdmin(ModelAdmin):
 ...
 menu_order = 100 
 ... 

class PizzaGroup(ModelAdminGroup):
 menu_label = "Pizzas" 
 menu_icon = "pick"
 menu_order = 500 
 items = (PizzaAdmin, ToppingAdmin)


modeladmin_register(PizzaGroup)
</code></pre>
 </div>
 
 In the
 <code>
 PizzaGroup
 </code>
 class, we have some of the same attributes:
 
 <ul>
 <li>
 <code>
 menu_label
 </code>
 : We set what we want this group of related models to be called in the Wagtail admin menu
 </li>
 <li>
 <code>
 menu_icon
 </code>
 : Which icon we want to use for this menu
 </li>
 <li>
 <code>
 menu_order
 </code>
 : Where we want this menu to appear in the sidebar, in relation to the other menu items
 </li>
 </ul>
 
 We also add a new attribute,
 <code>
 items
 </code>
 , where we list which
 <code>
 ModelAdmin
 </code>
 classes we want to be part of this group. In our case, we want
 <code>
 PizzaAdmin
 </code>
 and
 <code>
 ToppingAdmin
 </code>
 to be in this group, so we add those.
 
 
 Note the change we made to
 <code>
 menu_order
 </code>
 in
 <code>
 PizzaAdmin
 </code>
 and
 <code>
 ToppingAdmin
 </code>
 : Now those are set to
 <code>
 000
 </code>
 and
 <code>
 100
 </code>
 . When the
 <code>
 ModelAdmin
 </code>
 classes will be part of a group, set the
 <code>
 menu_order
 </code>
 how you want them to relate to each other, not to the other menu items in the Wagtail admin. Then set the
 <code>
 menu_order
 </code>
 for the
 <code>
 ModelAdminGroup
 </code>
 class to the proper value for the order you want it to appear in the side menu in the admin.
 
 
 Then we register the whole group, instead of the
 <code>
 ModelAdmin
 </code>
 classes individually, to the Wagtail admin. When we reload the admin, we see this:
 
 </div>
</div>
<div class="block-content">
 
 
 <img alt="pizzas_group_admin.png" class="richtext-image full-width" height="300" src="https://sfo2.digitaloceanspaces.com/revsys-com-prod-media/images/pizzas_group_admin.width-800.png" width="800"/>
 
 
</div>
<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 
 On the far left, there is a new menu item
 
 Pizzas
 
 that expands a submenu. The submenu contains links to the admin interfaces for
 
 Pizzas
 
 and
 
 Toppings
 
 !
 
 
 
 Note
 
 : If you have the Django admin enabled and have your models already in the Django admin, this doesn't disable them from the regular Django admin. You are free to access your models in both the Wagtail admin and the Django admin, or at this point you can choose to remove your models from the Django admin (or disable the Django admin altogether, if you prefer).
 
 <h2>
 Helpful Links
 </h2>
 <ul>
 <li>
 <a href="http://docs.wagtail.io/en/v2.6.1/reference/contrib/modeladmin/">
 Wagtail docs on ModelAdmin
 </a>
 </li>
 <li>
 <a href="https://thegrouchy.dev/general/2015/12/06/wagtail-streamfield-icons.html">
 A List of Wagtail StreamField Icons
 </a>
 </li>
 </ul>
 
 
 Special thanks to Jeff Triplett and Jacob Burch for their help with this post.
 
 
 </div>
</div>

from django.db import models 


class Pizza(models.Model):

    name = models.CharField(max_length=30)
    toppings = models.ManyToManyField("Topping")


class Topping(models.Model):

    name = models.CharField(max_length=30)

# settings.py 

INSTALLED_APPS = [
    ...
    "wagtail.contrib.modeladmin",
]

# wagtail_hooks.py

from wagtail.contrib.modeladmin.options import ModelAdmin, modeladmin_register 

from .models import Pizza


class PizzaAdmin(ModelAdmin):
    model = Pizza 
    menu_label = "Pizza"  
    menu_icon = "pick" 
    menu_order = 200 
    add_to_settings_menu = False 
    exclude_from_explorer = False 
    list_display = ("name",)
    list_filter = ("toppings",)
    search_fields = ("name",)


modeladmin_register(PizzaAdmin)

# wagtail_hooks.py

from wagtail.contrib.modeladmin.options import (
    ModelAdmin, 
    ModelAdminGroup, 
    modeladmin_register 
)

from .models import Pizza, Topping


class PizzaAdmin(ModelAdmin):
    ...
    menu_order = 000 
    ...


class ToppingAdmin(ModelAdmin):
    model = Topping 
    menu_label = "Toppings"  
    menu_icon = "edit" 
    menu_order = 100 
    add_to_settings_menu = False 
    exclude_from_explorer = False 
    list_display = ("name",)
    search_fields = ("name",)

# wagtail_hooks.py

from wagtail.contrib.modeladmin.options import (
    ModelAdmin, 
    ModelAdminGroup, 
    modeladmin_register 
)

from .models import Pizza, Topping


class PizzaAdmin(ModelAdmin):
    ...
    menu_order = 000 
    ...


class ToppingAdmin(ModelAdmin):
    ...
    menu_order = 100 
    ... 

class PizzaGroup(ModelAdminGroup):
    menu_label = "Pizzas" 
    menu_icon = "pick"
    menu_order = 500 
    items = (PizzaAdmin, ToppingAdmin)


modeladmin_register(PizzaGroup)

  
  
   In the
   
    PizzaGroup
   
   class, we have some of the same attributes:
  
  
   
    
     menu_label
    
    : We set what we want this group of related models to be called in the Wagtail admin menu
   
   
    
     menu_icon
    
    : Which icon we want to use for this menu
   
   
    
     menu_order
    
    : Where we want this menu to appear in the sidebar, in relation to the other menu items
   
  
  
   We also add a new attribute,
   
    items
   
   , where we list which
   
    ModelAdmin
   
   classes we want to be part of this group. In our case, we want
   
    PizzaAdmin
   
   and
   
    ToppingAdmin
   
   to be in this group, so we add those.
  
  
   Note the change we made to
   
    menu_order
   
   in
   
    PizzaAdmin
   
   and
   
    ToppingAdmin
   
   : Now those are set to
   
    000
   
   and
   
    100
   
   . When the
   
    ModelAdmin
   
   classes will be part of a group, set the
   
    menu_order
   
   how you want them to relate to each other, not to the other menu items in the Wagtail admin. Then set the
   
    menu_order
   
   for the
   
    ModelAdminGroup
   
   class to the proper value for the order you want it to appear in the side menu in the admin.
  
  
   Then we register the whole group, instead of the
   
    ModelAdmin
   
   classes individually, to the Wagtail admin. When we reload the admin, we see this:
  
 


 
 
 
 
 


 
  
   On the far left, there is a new menu item
   
    Pizzas
   
   that expands a submenu. The submenu contains links to the admin interfaces for
   
    Pizzas
   
   and
   
    Toppings
   
   !
  
  
   
    Note
   
   : If you have the Django admin enabled and have your models already in the Django admin, this doesn't disable them from the regular Django admin. You are free to access your models in both the Wagtail admin and the Django admin, or at this point you can choose to remove your models from the Django admin (or disable the Django admin altogether, if you prefer).
  
  
   Helpful Links
  
  
   
    
     Wagtail docs on ModelAdmin
    
   
   
    
     A List of Wagtail StreamField Icons
    
   
  
  
   
    Special thanks to Jeff Triplett and Jacob Burch for their help with this post.

When working with Wagtail, you might find that you're using Wagtail Page models for some of your database models, but regular Django models for others. In this post, learn how to add your Django models to the Wagtail admin to make managing multiple types of models easier.

How to Add Django Models to the Wagtail Admin

On a recent project, we needed to use different serializers for GET vs POST/PUT/PATCH requests to our API. Read on to learn how we used a mixin to accomplish this goal.

<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 
 
 Versions
 
 :
 
 <ul>
 <li>
 Python 3.7
 </li>
 <li>
 Django 2.2
 </li>
 <li>
 Django REST Framework 3.10
 </li>
 </ul>
 
 On a recent project, we needed to use different serializers for GET vs. POST/PUT/PATCH requests to our Django REST Framework API. In our case, this was because the GET serializer contained a lot of nested data; for example, it contained expanded fields from other serializers to foreign-key relationships. The requests to update data via the API, though, didn't need these expanded fields.
 
 
 The first way we approached using different serializers for read and update actions was to override
 <code>
 get_serializer_class()
 </code>
 on each viewset to decide which serializer to return depending on the action in the request. We returned the "read" serializer for
 <code>
 list
 </code>
 and
 <code>
 retrieve
 </code>
 actions, and the "update" serializer for everything else. (The full list of API actions is
 <a href="https://github.com/encode/django-rest-framework/blob/335054a5d36b352a58286b303b608b6bf48152f8/rest_framework/schemas/coreapi.py#L39">
 in the DRF codebase
 </a>
 .) But we wound up repeating ourselves across several viewsets, so we wrote a mixin to take care of some of this work for us!
 
 
 A mixin is a Python class that contains custom attributes and methods (
 <a href="https://easyaspython.com/mixins-for-fun-and-profit-cb9962760556?gi=3875fd6a6ff8">
 more explanation
 </a>
 ). It's not very useful on its own, but when it's inherited into a class, that class has access to the mixin's special attributes and methods.
 
 
 This was our mixin:
 
 <div class="codehilite">
 <pre><code>class ReadWriteSerializerMixin(object):
 """
 Overrides get_serializer_class to choose the read serializer
 for GET requests and the write serializer for POST requests.

 Set read_serializer_class and write_serializer_class attributes on a
 viewset. 
 """

 read_serializer_class = None
 write_serializer_class = None

 def get_serializer_class(self): 
 if self.action in ["create", "update", "partial_update", "destroy"]:
 return self.get_write_serializer_class()
 return self.get_read_serializer_class()

 def get_read_serializer_class(self):
 assert self.read_serializer_class is not None, (
 "'%s' should either include a `read_serializer_class` attribute,"
 "or override the `get_read_serializer_class()` method."
 % self.__class__.__name__
 )
 return self.read_serializer_class

 def get_write_serializer_class(self):
 assert self.write_serializer_class is not None, (
 "'%s' should either include a `write_serializer_class` attribute,"
 "or override the `get_write_serializer_class()` method."
 % self.__class__.__name__
 )
 return self.write_serializer_class
</code></pre>
 </div>
 
 This mixin defines two new attributes,
 <code>
 read_serializer_class
 </code>
 and
 <code>
 write_serializer_class
 </code>
 . Each attribute has a corresponding method to catch the error where the mixin is being used, but those attributes haven't been set. The
 <code>
 get_*_serializer_class()
 </code>
 methods will raise an
 <code>
 AssertionError
 </code>
 if your viewset hasn't set the appropriate attribute or overridden the necessary method.
 
 
 The
 <code>
 get_serializer_class
 </code>
 method makes the final decision on which serializer to use. For the "update" actions to the API, it returns
 <code>
 write_serializer_class
 </code>
 ; otherwise it returns
 <code>
 read_serializer_class
 </code>
 .
 
 
 The mixin gets used in a viewset like this:
 
 <div class="codehilite">
 <pre><code>from rest_framework import viewsets

from .mixins import ReadWriteSerializerMixin
from .models import MyModel
from .serializers import ModelReadSerializer, ModelWriteSerializer


class MyModelViewSet(ReadWriteSerializerMixin, viewsets.ModelViewSet):
 queryset = MyModel.objects.all() 
 read_serializer_class = ModelReadSerializer 
 write_serializer_class = ModelWriteSerializer
</code></pre>
 </div>
 
 Now the viewset
 <code>
 MyModelViewSet
 </code>
 has access to the attributes and methods from the mixin
 <code>
 ReadWriteSerializerMixin
 </code>
 . This means that when a call is made to the API that uses
 <code>
 MyModelViewSet
 </code>
 , the
 <code>
 get_serializer_class()
 </code>
 method from
 <code>
 ReadWriteSerializerMixin
 </code>
 will automatically be called and will decide, based on the kind of API request being made, which serializer to use. If we needed to make even more granular decisions about the serializer returned (maybe we want to use a more limited serializer for a
 <code>
 list
 </code>
 request and one with more data in a
 <code>
 retrieve
 </code>
 request), then our viewset can override
 <code>
 get_write_serializer_class()
 </code>
 to add that logic.
 
 
 Note: Custom DRF actions will contain actions that aren't part of the DRF list of accepted actions (because they are custom actions you're creating), so when you call
 <code>
 get_serializer_class
 </code>
 from inside your action method, it will return whatever your "default" serializer class is. In the example above, the "default" serializer is the
 <code>
 read_serializer_class
 </code>
 because it's what we return when we fall through the other conditional.
 
 
 Depending on your action, you will want to override
 <code>
 get_serializer_class
 </code>
 to change your default method or explicitly account for your custom action.
 
 
 Mixins are a DRY (Don't Repeat Yourself) way to add functionality that you wind up needing to use across several viewsets. We hope you get to experiment with using them soon!
 
 
 
 Thanks to Jeff Triplett for his help with this post.
 
 
 </div>
</div>

class ReadWriteSerializerMixin(object):
    """
    Overrides get_serializer_class to choose the read serializer
    for GET requests and the write serializer for POST requests.

    Set read_serializer_class and write_serializer_class attributes on a
    viewset. 
    """

    read_serializer_class = None
    write_serializer_class = None

    def get_serializer_class(self):        
        if self.action in ["create", "update", "partial_update", "destroy"]:
            return self.get_write_serializer_class()
        return self.get_read_serializer_class()

    def get_read_serializer_class(self):
        assert self.read_serializer_class is not None, (
            "'%s' should either include a `read_serializer_class` attribute,"
            "or override the `get_read_serializer_class()` method."
            % self.__class__.__name__
        )
        return self.read_serializer_class

    def get_write_serializer_class(self):
        assert self.write_serializer_class is not None, (
            "'%s' should either include a `write_serializer_class` attribute,"
            "or override the `get_write_serializer_class()` method."
            % self.__class__.__name__
        )
        return self.write_serializer_class

from rest_framework import viewsets

from .mixins import ReadWriteSerializerMixin
from .models import MyModel
from .serializers import ModelReadSerializer, ModelWriteSerializer


class MyModelViewSet(ReadWriteSerializerMixin, viewsets.ModelViewSet):
    queryset = MyModel.objects.all() 
    read_serializer_class = ModelReadSerializer 
    write_serializer_class = ModelWriteSerializer

  
  
   Now the viewset
   
    MyModelViewSet
   
   has access to the attributes and methods from the mixin
   
    ReadWriteSerializerMixin
   
   . This means that when a call is made to the API that uses
   
    MyModelViewSet
   
   , the
   
    get_serializer_class()
   
   method from
   
    ReadWriteSerializerMixin
   
   will automatically be called and will decide, based on the kind of API request being made, which serializer to use. If we needed to make even more granular decisions about the serializer returned (maybe we want to use a more limited serializer for a
   
    list
   
   request and one with more data in a
   
    retrieve
   
   request), then our viewset can override
   
    get_write_serializer_class()
   
   to add that logic.
  
  
   Note: Custom DRF actions will contain actions that aren't part of the DRF list of accepted actions (because they are custom actions you're creating), so when you call
   
    get_serializer_class
   
   from inside your action method, it will return whatever your "default" serializer class is. In the example above, the "default" serializer is the
   
    read_serializer_class
   
   because it's what we return when we fall through the other conditional.
  
  
   Depending on your action, you will want to override
   
    get_serializer_class
   
   to change your default method or explicitly account for your custom action.
  
  
   Mixins are a DRY (Don't Repeat Yourself) way to add functionality that you wind up needing to use across several viewsets. We hope you get to experiment with using them soon!
  
  
   
    Thanks to Jeff Triplett for his help with this post.

Using Different Read and Write Serializers in Django REST Framework

Python 3.7 introduced dataclasses, which design is based on the &quot;attrs&quot; library. This article will show the way I use dataclasses and attrs, why I think you should use both, and why I think attrs is still very relevant.

def render_document(title: str, caption: str, data: List[Dict[str, Any]]):
    return template.render({
        "title" : title,
        "caption": caption,
        "data": data,
})

{
    "title": "My Table",
    "caption": "2019 Earnings",
    "data": [
        {"Period": "QT1", "Europe": 500, "USA": 467},
        {"Period": "QT2", "Europe": 345, "USA": 765},
    ]
}

def render_document(title: str, description: str, status: str, tables: List[Dict[str, Any]]):
    return template.render({
        "title": title,
        "description": description,
        "status": status,
        "tables": tables,
    })

@dataclass
class Table:
    title: str
    data: List[Dict[str, Any]]
    caption: str = ""

def render_document(title: str, description: str, tables: List[Table]):
    return template.render({
        "title": title,
        "description": description,
        "tables": tables,
    })

@attr.s
class Table(object):
    title: str = attr.ib(validator=attr.validators.instance_of(str))  # don't you pass no bytes!
    data: List[Dict[str, Any]] = attr.ib(validator=...)
    description: str = attr.ib(validator=attr.validators.instance_of(str), default="")


def render_document(title: str, description: str, tables: List[Table]):
    return template.render({
        "title": title,
        "description": description,
        "tables": tables,
    })

@dataclass
class Table:
    title: str
    data: List[Dict[str, Any]]
    caption: str = ""

@attr.s
class Document(object):
    status: str = attr.ib(validators=attr.validators.in_(
        ["draft", "in review", "approved"]
    ))
    tables: List[Table] = attr.ib(default=[])

def render_report(self, title: str, documents: List[Document]):
    return template.render({
        "title": title,
        "documents": documents,
    })

  
  
   Note how I am validating that
   
    Document.status
   
   is one of the allowed values. This comes particularly handy when you're building abstractions on top of Django models with a field that uses
   
    choices
   
   . Dataclasses can't do that.
  
  
   A couple of patterns I keep finding myself in are the following:
  
  
   
    Write a function that accepts some arguments
   
   
    Group some of the arguments into a
    
     tuple
    
   
   
    Hm, I want field names ->
    
     namedtuple
    
    .
   
   
    Hm, I want types ->
    
     dataclass
    
    .
   
   
    Hm, I want validation ->
    
     attrs
    
    .
   
  
  
   Another situation that happens quite often is this:
  
  
   
    write a function that accepts some arguments
   
   
    add typing so my IDE can help me out
   
   
    oh, by the way, it needs to support a list of those things, not just one at a time!
   
   
    refactor to use dataclasses
   
   
    This argument can only be one of those values, or
    
     
      I ask myself: How do I make sure other developers are passing the right type and/or values?
     
    
   
   
    switch to attrs
   
  
  
   Sometimes I stop at the dataclasses. Lots of times I get to the attrs step.
  
  
   And sometimes, this happens:
1. one half of this legacy codebase uses
   
    -1
   
   as special value for
   
    False
   
   , that other half uses
   
    False
   
   . Switch to
   
    attr.s
   
   so I can use
   
    converter=
   
   to normalize.
  
  
   Comparison
  
  
   The two libraries do appear very similar. To get a clearer picture of how they compare, I've made a table of the features I use most:
  
  
   
    
     
      feature
     
     
      dataclasses
     
     
      attrs
     
    
   
   
    
     
      frozen
     
     
      ✓
     
     
      ✓
     
    
    
     
      defaults
     
     
      ✓
     
     
      ✓
     
    
    
     
      totuple
     
     
      ✓
     
     
      ✓
     
    
    
     
      todict
     
     
      ✓
     
     
      ✓
     
    
    
     
      validators
     
     
      ✗
     
     
      ✓
     
    
    
     
      converters
     
     
      ✗
     
     
      ✓
     
    
    
     
      slotted classes
     
     
      ✗
     
     
      ✓
     
    
   
  
  
   As you can see, there's a lot of overlap. But the additional features on
   
    attrs
   
   provide functionality that I need more often than not.
  
  
   When to use dataclasses
  
  
   Dataclasses are just about the "shape" of the data.
Choose dataclasses if:
  
  
   
    You don't care about values in the fields, only their type
   
   
    adding a dependency is not trivial
   
  
  
   When to use attrs
  
  
   attrs is about the shape
   
    and
   
   the values.
Choose attrs if:
  
  
   
    you want to validate values. A common case would be the equivalent of a ChoiceField.
   
   
    you want to normalize, or sanitize the input
   
   
    whenever you want more formalization than dataclasses alone can offer
   
   
    you are concerned about memory and performances.
    
     attrs
    
    can create
    
     slotted classes
    
    , which are
    
     optimized by CPython
    
    .
   
  
  
   I often find myself using dataclasses and later switching to attr.s because the requirements changed or I find out I need to guard against some particular value. I think that's a normal aspect of developing software and what I call "continuous refactoring".
  
  
   Why I like dataclasses
  
  
   I'm glad dataclasses have been added to the standard library, and I think it's a beneficial addition. It's a very convenient thing to have at your disposal whenever you need.
  
  
   For one, it will encourage a more structured style of programming from the beginning.
  
  
   But I think the most compelling case is a practical one. Some high-risk corporate environments (eg: financial institutions) require every package to be vetted (with good reason: we've already had incidents of malicious code in libraries). That means that adding attrs is not as simple as adding a line to your
   
    requirements.txt
   
   , and will involve waiting on approval from your corpops team. Those developers can use dataclasses right away and their code will immediately benefit from using more formalized data types.
  
  
   Why I like attrs
  
  
   Most people don't work in such strictly-controlled environments.
  
  
   And sure, sometimes you don't need all the features from attrs, but it doesn't hurt having them.
  
  
   More often than not, I end up needing them anyway, as I formalize more and more of my code's API. Dataclasses only gets half-way of where I want to go.
  
  
   Conclusion
  
  
   I think dataclasses encompass only a subset of what attrs has to offer. Admittedly, it is a
   
    big
   
   subset. But the features that are not covered are important enough and needed often enough that they make attrs not only still relevant and useful, but also necessary.
  
  
   In my mind, using both allows developers to progressively refactor their code, moving the contracts amongst functions from loosely-defined arguments all the way up to formally described data structures as the requirements of the app stabilize over time.
  
  
   One nice effect of having dataclasses is that now developers are more incentivized to refactor their code toward more formalization. At some point dataclasses is not going to be enough, and that's when developers will refactor to use attrs. In this way, dataclasses actually acts as an
   
    introduction
   
   to attrs. I wouldn't be surprised if attrs becomes
   
    more
   
   popular thanks to dataclasses.
  
  
   References
  
  
   
    
     Documentation on dataclasses
    
    +
    
     PEP557
    
   
   
    
     Documentation on attrs
    
   
   
    
     "Why not dataclasses" on attrs' docs
    
   
  
  
   Acknowledgments + Thanks
  
  
   I want to thank the following people for revising drafts and providing input and insights:
  
  
   
    Hynek Schlawack
   
   
    Jacob Kaplan-Moss
   
   
    Jacob Burch
   
   
    Jeff Triplett

Python 3.7 introduced dataclasses, which design is based on the "attrs" library. This article will show the way I use dataclasses and attrs, why I think you should use both, and why I think attrs is still very relevant.

Dataclasses and attrs: when and why

How to smartly keep a set of Django ORM objects in order for drag-n-drop like situations.

class Task(models.Model):
 name = models.CharField(max_length=100)

class Step(models.Model):
 task = models.ForeignKey(Task, related_name='steps', on_delete=models.CASCADE)
 name = models.CharField(max_length=100)
 order = models.IntegerField(default=1)

 objects = StepManager()

from django.db import models, transaction
from django.db.models import F

class StepManager(models.Manager):
 """ Manager to encapsulate bits of business logic """

 def move(self, obj, new_order):
 """ Move an object to a new order position """

 qs = self.get_queryset()

 with transaction.atomic():
 if obj.order > int(new_order):
 qs.filter(
 task=obj.task,
 order__lt=obj.order,
 order__gte=new_order,
 ).exclude(
 pk=obj.pk
 ).update(
 order=F('order') + 1,
 )
 else:
 qs.filter(
 task=obj.task,
 order__lte=new_order,
 order__gt=obj.order,
 ).exclude(
 pk=obj.pk,
 ).update(
 order=F('order') - 1,
 )

 obj.order = new_order
 obj.save()

Step.objects.move(step_3_obj, 5)

from django.db import models, transaction
from django.db.models import F, Max

class StepManager(models.Manager):
 # ... previous code here ...

 def create(self, **kwargs):
 instance = self.model(**kwargs)

 with transaction.atomic():
 # Get our current max order number
 results = self.filter(
 task=instance.task
 ).aggregate(
 Max('order')
 )

 # Increment and use it for our new object
 current_order = results['order__max']
 if current_order is None:
 current_order = 0

 value = current_order + 1
 instance.order = value
 instance.save()

 return instance

class Step(models.Model):

 # ... original fields and manager definitions ... 

 class Meta:
 index_together = ('task', 'order')

from rest_framework import viewsets, status
from rest_framework.decorators import action
from rest_framework.response import Response
from django_filters.rest_framework import DjangoFilterBackend

from . import models
from . import serializers

class StepViewSet(viewsets.ModelViewSet):
 queryset = models.Step.objects.all().order_by('order')
 serializer_class = serializers.StepSerializer
 filter_backends = (DjangoFilterBackend, )
 filter_fields = ('task', )

 @action(methods=['post'], detail=True)
 def move(self, request, pk):
 """ Move a single Step to a new position """
 obj = self.get_object()
 new_order = request.data.get('order', None)

 # Make sure we received an order 
 if new_order is None:
 return Response(
 data={'error': 'No order given'},
 status=status.HTTP_400_BAD_REQUEST,
 )

 # Make sure our new order is not below one
 if int(new_order) < 1:
 return Response(
 data={'error': 'Order cannot be zero or below'},
 status=status.HTTP_400_BAD_REQUEST,
 )

 models.Step.objects.move(obj, new_order)

 return Response({'success': True, 'order': new_order})

 
 
 Where these endpoints reside is going to depend on how you configure things in your Django project, but assuming you're using a DRF
 
 SimpleRouter
 
 and have things setup on
 
 /api/
 
 your URL patterns are going to be something like:
 
 
 
 
 GET /api/steps/
 
 for a list of all steps
 
 
 
 GET /api/steps/?task=<task_id>
 
 for all Steps by Task (in Step order)
 
 
 
 POST /api/steps/<step_id>/move/
 
 with a JSON body of
 
 {'order': <int>}
 
 of the that Step's new order position.
 
 
 
 
 NOTE
 
 This API example excludes any authentication and authorization concerns which you will need to address before using in production. You don't want just anyone reordering your steps!
 
 
 Hopefully, you've found this informative and can put this pattern to use
in your next project. If you need help implementing things of this
nature at your company
 
 reach out to us
 
 to check on our
availability.

Keeping Django Models Ordered

ManyToManyFields confuse a lot of people. The way you relate objects to each other is just different enough from dealing with ForeignKeys and just uncommon enough in day-to-day Django development that it&#x27;s easy to forget all the little tricks for dealing with them.

from django.db import models 


class Pizza(models.Model):

 name = models.CharField(max_length=30)
 toppings = models.ManyToManyField('Topping')

 def __str__(self):
 return self.name


class Topping(models.Model):

 name = models.CharField(max_length=30)

 def __str__(self):
 return self.name

>> from pizzas.models import Pizza, Topping
>> hawaiian_pizza = Pizza.objects.create(name='Hawaiian')
>> pineapple = Topping(name='pineapple')
>> hawaiian_pizza.toppings.add(pineapple)
Traceback (most recent call last):
...
ValueError: Cannot add "<Topping: pineapple>": instance is on database "default", 
value is on database "None"
>> 

>> pineapple.save() 
>> hawaiian_pizza.toppings.add(pineapple)
>> hawaiian_pizza.toppings.all()
<QuerySet [<Topping: pineapple>]>

>> pepperoni = Topping.objects.create(name='pepperoni')
>> pepperoni_pizza = Pizza(name='Pepperoni')
>> pepperoni_pizza.toppings.add(pepperoni)
Traceback (most recent call last):
...
ValueError: "<Pizza: Pepperoni>" needs to have a value for field "id" before this many-to-many 
relationship can be used.

>> hawaiian_pizza.toppings.all()
<QuerySet [<Topping: pineapple>, <Topping: Canadian bacon>]>

>> canadian_bacon.pizzas.all()
Traceback (most recent call last):
...
AttributeError: 'Topping' object has no attribute 'pizzas'

>> canadian_bacon.pizza_set.all()
<QuerySet [<Pizza: Hawaiian>]>

class Pizza(models.Model):
 ...
 toppings = models.ManyToManyField('Topping', related_name='pizzas')

>> canadian_bacon.pizza_set.all()
<QuerySet [<Pizza: Hawaiian>]>

>> canadian_bacon.pizzas.all()
<QuerySet [<Pizza: Hawaiian>]>

>> cheese_pizza = Pizza.objects.create(name='Cheese')
>> mozzarella = Topping.objects.create(name='mozzarella')
>> mozzarella.pizzas.add(cheese_pizza)
>> mozzarella.pizzas.all()
<QuerySet [<Pizza: Cheese>]>

>> Pizza.objects.filter(toppings__name__startswith='p')
<QuerySet [<Pizza: Pepperoni Pizza>, <Pizza: Hawaiian Pizza>]>

>> Topping.objects.filter(pizzas__name__contains='Hawaiian')
<QuerySet [<Topping: pineapple>, <Topping: Canadian bacon>]>

>> pepperoni_pizza.toppings.all()
<QuerySet [<Topping: pepperoni>]>
>> pepperoni_pizza.toppings.add(pepperoni)
>> pepperoni_pizza.toppings.all()
<QuerySet [<Topping: pepperoni>]>

ValueError: Cannot alter field pizzas.Pizza.toppings into pizzas.Pizza.toppings - 
they are not compatible types (you cannot alter to or from M2M fields, or add or remove through= on M2M fields)

class ToppingAmount(models.Model):

 REGULAR = 1
 DOUBLE = 2
 TRIPLE = 3
 AMOUNT_CHOICES = (
 (REGULAR, 'Regular'),
 (DOUBLE, 'Double'),
 (TRIPLE, 'Triple'),
 )

 pizza = models.ForeignKey('Pizza', related_name='topping_amounts', on_delete=models.SET_NULL, null=True)
 topping = models.ForeignKey('Topping', related_name='topping_amounts', on_delete=models.SET_NULL, null=True, blank=True)
 amount = models.IntegerField(choices=AMOUNT_CHOICES, default=REGULAR)

class Pizza(models.Model):
 ...
 toppings = models.ManyToManyField('Topping', through='ToppingAmount', related_name='pizzas')

>> super_pep = Pizza.objects.create(name='Super Pepperoni')
>> pepperoni = Topping.objects.create(name='pepperoni')
>> super_pep.toppings.add(pepperoni)
Traceback (most recent call last):
...
AttributeError: Cannot use add() on a ManyToManyField which specifies an intermediary model. 
Use pizzas.ToppingAmount's Manager instead.

>>> super_pep_amount = ToppingAmount.objects.create(pizza=super_pep, topping=pepperoni, amount=ToppingAmount.DOUBLE)

>> super_pep.toppings.all()
<QuerySet [<Topping: pepperoni>]>

>> pepperoni.pizzas.all()
<QuerySet [<Pizza: Super Pepperoni>]>

for top_amt in ToppingAmount.objects.filter(pizza=super_pep):
 print(top_amt.topping.name, top_amt.get_amount_display())

pepperoni Double

for top_amt in super_pep.topping_amounts.all():
 print(top_amt.topping.name, top_amt.get_amount_display()) 

pepperoni Double

for topping in super_pep.toppings.all():
 for top_amt in topping.topping_amount.all():
 print(topping.name, top_amt.amount, top_amt.get_amount_display())

pepperoni 2 Double

 
 
 You could extend this
 
 ToppingAmount
 
 through model to hold information about the left and right halves of the pizza, or notes about topping preparation ("put peppers under cheese"). You can also add methods to the through model or the source/target models to more easily access some of the topping amount information.
 
 
 A through model is also useful for relationships between players and teams; the through model could contain information about the players' positions, jersey numbers, and dates they joined the team. A through model joining movie theatres and films could contain the number of screens the film is showing on and the start and end run dates. Students' relationships to their Degree Programs could track information like GPA, whether the program is the student's major or minor, whether it's a double major, and start/end semesters the student was in a program.
 
 
 
 Further Reading
 
 
 
 The
 
 many-to-many relationships
 
 section of the Django docs contains a lot of great examples of using
 
 ManyToManyField
 
 .
 
 
 The explanation of
 
 
 ManyToManyField
 
 
 in the model field reference.
 
 
 Jacob Kaplan-Moss also has a
 
 great set of examples
 
 of using a many-to-many relationship with a custom "through" model.
 
 
 
 
 Thank you to
 
 Monique Murphy
 
 and
 
 Jeff Triplett
 
 for their assistance.

ManyToManyFields confuse a lot of people. The way you relate objects to each other is just different enough from dealing with ForeignKeys and just uncommon enough in day-to-day Django development that it's easy to forget all the little tricks for dealing with them.

Tips for Using Django's ManyToManyField

A very common pattern in a Django Project is to have some kind of &#x27;model-type&#x27; relationship, where you have some kind of object that can only belong to one of the types defined in the database.

<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 <h1>
 Slugs as primary keys
 </h1>
 
 A very common pattern I've seen in Django Project is to have some kind of 'model-type' relationship, where you have some kind of object that can only belong to one of the types defined in the database.
 
 
 A typical implementation would look something like this:
 
 <div class="codehilite">
 <pre><code>class EventType(models.Model):
 slug = models.SlugField(unique=True)
 name = models.CharField(max_length=100)


class Event(models.Model):
 type = models.ForeignKey(EventType, on_delete=models.CASCADE)
 title = models.CharField(max_length=100)
 day = models.DateField()
</code></pre>
 </div>
 
 A subtle issue with this implementation is that you may have to query the DB for an
 <code>
 EventType
 </code>
 when creating an
 <code>
 Event
 </code>
 :
 
 <div class="codehilite">
 <pre><code>work_type = EventType.objects.get(slug='work')
Event.objects.create(
 type=work_type,
 title='release',
 day=datetime(2018, 12, 31),
)
</code></pre>
 </div>
 
 In other words, the minimum amount of queries to create an
 <code>
 Event
 </code>
 is 2: a
 <code>
 SELECT
 </code>
 for fetching the type, and the
 <code>
 INSERT
 </code>
 that saves the instance.
 
 
 If somehow you already have the
 <code>
 pk
 </code>
 of the
 <code>
 EventType
 </code>
 (for example, it might come from an API payload or from the URL), then you can easily avoid the lookup by setting the primary key directly into the
 <code>
 type_id
 </code>
 column:
 
 <div class="codehilite">
 <pre><code>def post(self, request, *args, **kwargs):
 type_id = self.kwargs['pk']
 Event.objects.create(
 type_id=type_id,
 title='release',
 day=datetime(2018, 12, 31),
 )
</code></pre>
 </div>
 
 But dealing directly with column names is
 <a href="https://docs.djangoproject.com/en/2.1/ref/models/fields/#database-representation">
 discouraged by the docs
 </a>
 
 <blockquote>
 
 However, your code should never have to deal with the database column name
 
 </blockquote>
 
 We can get around this by instantiating an
 <code>
 EventType
 </code>
 instance with just the primary key:
 
 <div class="codehilite">
 <pre><code>def post(self, request, *args, **kwargs):
 type_id = self.kwargs['pk']
 Event.objects.create(
 type=EventType(pk=type_id),
 title='release',
 day=datetime(2018, 12, 31),
 )
</code></pre>
 </div>
 
 But this requires us to know the numerical
 <code>
 id
 </code>
 beforehand.
 
 
 We already have
 <code>
 slug
 </code>
 as source of uniqueness for the
 <code>
 event_eventtypes
 </code>
 table, and it's URL-friendly. We could just use that as the
 <code>
 EventType
 </code>
 primary key.
 
 <div class="codehilite">
 <pre><code>class EventType(models.Model):
 slug = models.SlugField(primary_key=True)
 name = models.CharField(max_length=100)


class Event(models.Model):
 type = models.ForeignKey(
 EventType, 
 on_delete=models.CASCADE,
 )
 title = models.CharField(max_length=100)
 day = models.DateField()
</code></pre>
 </div>
 <div class="codehilite">
 <pre><code>def post(self, request, *args, **kwargs):
 type_slug = self.kwargs['slug']
 Event.objects.create(
 type=EventType(pk=type_slug),
 title='release',
 day=datetime(2018, 12, 31),
 )
</code></pre>
 </div>
 
 This also allows to easily create
 <code>
 Event
 </code>
 s of a specific type without having to fetching any
 <code>
 EventType
 </code>
 . This is especially useful in data migrations, or tests:
 
 <div class="codehilite">
 <pre><code>types = ['home', 'work', 'community']
for type_slug in types:
 Event.objects.create(
 type=EventType(pk=type_slug),
 title='release',
 day=datetime(2018, 12, 31),
 )
</code></pre>
 </div>
 
 In cases like this, you may want to consider using a slug as primary key, rather than the default integer-based default. It's more performant and just as straight forward.
 
 
 There are some caveat to consider, though: You won't be able to modify the slug from the Django admin.
 
 
 Another thing to consider is that changing the type of your primary key is a one-way road: once you made the slug your pk, you won't be able to convert it back to a regular
 <code>
 AutoField
 </code>
 .
 
 </div>
</div>

class EventType(models.Model):
    slug = models.SlugField(unique=True)
    name = models.CharField(max_length=100)


class Event(models.Model):
    type = models.ForeignKey(EventType, on_delete=models.CASCADE)
    title = models.CharField(max_length=100)
    day = models.DateField()

work_type = EventType.objects.get(slug='work')
Event.objects.create(
    type=work_type,
    title='release',
    day=datetime(2018, 12, 31),
)

def post(self, request, *args, **kwargs):
    type_id = self.kwargs['pk']
    Event.objects.create(
        type_id=type_id,
        title='release',
        day=datetime(2018, 12, 31),
    )

def post(self, request, *args, **kwargs):
    type_id = self.kwargs['pk']
    Event.objects.create(
       type=EventType(pk=type_id),
       title='release',
       day=datetime(2018, 12, 31),
    )

class EventType(models.Model):
    slug = models.SlugField(primary_key=True)
    name = models.CharField(max_length=100)


class Event(models.Model):
    type = models.ForeignKey(
        EventType, 
        on_delete=models.CASCADE,
    )
    title = models.CharField(max_length=100)
    day = models.DateField()

def post(self, request, *args, **kwargs):
    type_slug = self.kwargs['slug']
    Event.objects.create(
       type=EventType(pk=type_slug),
       title='release',
       day=datetime(2018, 12, 31),
    )

types =  ['home', 'work', 'community']
for type_slug in types:
    Event.objects.create(
       type=EventType(pk=type_slug),
       title='release',
       day=datetime(2018, 12, 31),
    )

  
  
   In cases like this, you may want to consider using a slug as primary key, rather than the default integer-based default. It's more performant and just as straight forward.
  
  
   There are some caveat to consider, though: You won't be able to modify the slug from the Django admin.
  
  
   Another thing to consider is that changing the type of your primary key is a one-way road: once you made the slug your pk, you won't be able to convert it back to a regular
   
    AutoField
   
   .

A very common pattern in a Django Project is to have some kind of 'model-type' relationship, where you have some kind of object that can only belong to one of the types defined in the database.

Slugs as primary keys

It is often necessary to differentiate between an argument that has not been provided, and an argument provided with the value `None`. For that purpose, we create what&#x27;s called a &#x27;sentinel value&#x27;.

<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 
 Sometimes it is necessary to differentiate between an argument that has not
been provided, and an argument provided with the value
 <code>
 None
 </code>
 . For that purpose, we create what's called a 'sentinel value'.
 
 
 For example, let's assume you want to define a Field class. Field instances must have a value or declare a default,
and
 <code>
 None
 </code>
 could be a perfectly valid value to have:
 
 <div class="codehilite">
 <pre><code>class Field:
 def __init__(self, default=sentinel):
 self.value = default

 def set(self, value):
 self.value = value

 def get(self):
 if self.value is sentinel:
 raise ValueError("this field has no value!")

eula_accepted = Field()
eula_accepted.get() # raises `ValueError`

eula_accepted = Field(default=None)
eula_accepted.get() # doesn't raise. `None` means the EULA hasn't been neither accepted or reject yet.
</code></pre>
 </div>
 
 The most common approach is to declare the sentinel value with
 <code>
 object()
 </code>
 :
 
 <div class="codehilite">
 <pre><code>sentinel = object()
</code></pre>
 </div>
 
 This approach is the quickest and most common, but it has some issues. To quote
 <a href="https://mail.python.org/pipermail/python-ideas/2018-July/051852.html">
 Joseph Jevnik
 </a>
 :
 
 <blockquote>
 
 One is that they don't repr well so they make debugging harder. Another issue
is that they cannot be pickled or copied. You also cannot take a weak
reference to a sentinel which can break some caching code and makes
them harder to use.
 
 </blockquote>
 
 For example:
 
 <div class="codehilite">
 <pre><code>sentinel = object()

repr(sentinel)
# '&lt;object object at 0x10823e8d0&gt;'
</code></pre>
 </div>
 
 To work around this issue, some people create their own
 <code>
 Sentinel
 </code>
 class. But I've found a quicker way in the
 <code>
 unittest.mock
 </code>
 module.
 
 <div class="codehilite">
 <pre><code>from unittest.mock import sentinel

NotSet = sentinel.NotSet

repr(NotSet)
# 'sentinel.NotSet'
</code></pre>
 </div>
 
 If you don't feel like importing from
 <code>
 unittest
 </code>
 in your application code, you could install the
 <code>
 mock
 </code>
 package, or "hide it under the rug" by aliasing somewhere in your code base and importing it from there:
 
 <div class="codehilite">
 <pre><code># in `myproject.types`
from unittest.mock import sentinel
</code></pre>
 </div>
 <div class="codehilite">
 <pre><code># somewhere else in your project:
from myproject.types import sentinel
</code></pre>
 </div>
 
 An alternative to
 <code>
 unittest.mock.sentinel
 </code>
 is to declare your own sentinel class and use it as a value:
 
 <div class="codehilite">
 <pre><code>class NotSet:
 pass

# PS: Remember to use the class _itself_.
def fn(default=NotSet):
 pass
</code></pre>
 </div>
 
 This will give you a not really pretty, but useful enough
 <code>
 repr
 </code>
 :
 
 <div class="codehilite">
 <pre><code>repr(NotSet)
# "&lt;class '__main__.NotSet'&gt;"
</code></pre>
 </div>
 
 Of course, you could go one step further and declare your own
 <code>
 repr
 </code>
 :
 
 <div class="codehilite">
 <pre><code>class NotSet:
 def __repr__(self):
 return 'NotSet'

repr(NotSet)
# 'NotSet'
</code></pre>
 </div>
 
 Of all the options, I think using
 <code>
 unittest.mock.Sentinel
 </code>
 is my favorite. Importing from
 <code>
 unittest
 </code>
 in my application code is a compromise that I'm willing to make in exchange for having something ready to use.
 
 </div>
</div>

class Field:
 def __init__(self, default=sentinel):
 self.value = default

 def set(self, value):
 self.value = value

 def get(self):
 if self.value is sentinel:
 raise ValueError("this field has no value!")

eula_accepted = Field()
eula_accepted.get() # raises `ValueError`

eula_accepted = Field(default=None)
eula_accepted.get() # doesn't raise. `None` means the EULA hasn't been neither accepted or reject yet.

sentinel = object()

sentinel = object()

repr(sentinel)
# '<object object at 0x10823e8d0>'

from unittest.mock import sentinel

NotSet = sentinel.NotSet

repr(NotSet)
# 'sentinel.NotSet'

# in `myproject.types`
from unittest.mock import sentinel

# somewhere else in your project:
from myproject.types import sentinel

class NotSet:
 pass

# PS: Remember to use the class _itself_.
def fn(default=NotSet):
 pass

repr(NotSet)
# "<class '__main__.NotSet'>"

class NotSet:
 def __repr__(self):
 return 'NotSet'

repr(NotSet)
# 'NotSet'

 
 
 Of all the options, I think using
 
 unittest.mock.Sentinel
 
 is my favorite. Importing from
 
 unittest
 
 in my application code is a compromise that I'm willing to make in exchange for having something ready to use.

It is often necessary to differentiate between an argument that has not been provided, and an argument provided with the value `None`. For that purpose, we create what's called a 'sentinel value'.

Sentinel values in Python

In this post, you will learn how to create a Celery task inside a Django project in a Docker container. Sounds awesome, right?

proj/
├── app/
 ├── __init__.py
 ├── tasks.py
├── proj/
 ├── __init__.py
 ├── celery.py
 ├── settings.py
├── docker-compose.yml
├── Dockerfile
├── manage.py
└── requirements.txt

celery==4.2.1
redis==2.10.6

import os
from celery import Celery


os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'proj.settings')

app = Celery('proj')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()

@app.task(bind=True)
def debug_task(self):
 print('Request: {0!r}'.format(self.request))

from celery import shared_task


@shared_task
def hello():
 print(“Hello there!”) 

from .celery import app as celery_app

__all__ = ['celery_app']

>>> from app.tasks import hello
>>> hello()
Hello there! 

>>> hello.delay()
<AsyncResult: ba845cf3-e60b-4432-a9d8-9943621cb8a0>

CELERY_BROKER_URL = 'redis://redis:6379'
CELERY_RESULT_BACKEND = 'redis://redis:6379'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'

from celery.schedules import crontab

CELERY_BEAT_SCHEDULE = {
 'hello': {
 'task': 'app.tasks.hello',
 'schedule': crontab() # execute every minute
 }
}

version: '3'

services:
 db:
 image: postgres:9.6.5
 volumes:
 - postgres_data:/var/lib/postgresql/data/
 redis:
 image: "redis:alpine"
 web:
 build: .
 command: bash -c "python /code/manage.py migrate --noinput && python /code/manage.py runserver 0.0.0.0:8000"
 volumes:
 - .:/code
 ports:
 - "8000:8000"
 depends_on:
 - db
 - redis
 celery:
 build: .
 command: celery -A proj worker -l info
 volumes:
 - .:/code
 depends_on:
 - db
 - redis
 celery-beat:
 build: .
 command: celery -A proj beat -l info
 volumes:
 - .:/code
 depends_on:
 - db
 - redis

volumes:
 postgres_data:

redis:
 image: "redis:alpine"

depends_on:
 - db
 - redis

celery:
 build: .
 command: celery -A proj worker -l info
 volumes:
 - .:/code
 depends_on:
 - db
 - redis

celery-beat:
 build: .
 command: celery -A proj beat -l info
 volumes:
 - .:/code
 depends_on:
 - db
 - redis

celery_1 | [2018-02-01 22:44:00,957: WARNING/ForkPoolWorker-1] Hello there!

 
 
 And there you have it! For even more fun, you might try:
 
 
 
 Adding more complex tasks, like tasks that take arguments
 
 
 Setting up different schedules
 
 
 Setting up a non-periodic task
 
 
 
 What next?
 
 
 Review the
 
 Celery Tasks Checklist
 
 for a great introduction to Celery best practices. In particular, pay attention to:
 
 
 
 Set a
 
 retry_limit
 
 so that failed tasks don’t keep trying forever.
 
 
 Set a high default
 
 task_time_limit
 
 to avoid tasks that block the entire Celery worker.
 
 
 For tasks that need to take in a Django model object as a parameter, pass in a primary key and not the object itself.
 
 
 Do a lot of logging!
 
 
 
 You will also want to monitor your tasks for success or failure. A great tool for this is
 
 Flower
 
 , Celery’s monitoring tool. Flower will show you a dashboard of all your workers and tasks and let you drill down into specific tasks, show you task statistics, let you restart workers, and let you rate-limit tasks (among many other things).
 
 
 If you use an error-tracking system like
 
 Rollbar
 
 or
 
 Sentry
 
 , you can also set Celery up to report exceptions to those services.
 
 
 Further Reading
 
 
 
 
 Celery Task Checklist
 
 
 
 
 Message Queues and Async Workers
 
 , Frank Wiles
 
 
 
 Celery Docs - Periodic Tasks
 
 
 
 
 Celery Docs - First steps with Django
 
 
 
 
 Celery Docs - task_always_eager
 
 
 
 
 Celery Docs - Logging
 
 
 
 
 Celery 4 Periodic Task in Django
 
 , Yehan Djoehartono
 
 
 
 Asynchronous Tasks with Django and Celery
 
 (Celery 3.1.18)
 
 
 
 
 * Thanks to
 
 kurashu89
 
 for their correction on an earlier version of this article.

Celery and Django and Docker: Oh My!

django-test-plus has long been a useful helper library with Django. Now we&#x27;ve added some simple pytest fixture support to make it even more useful!

<div class="block-content">
 
 We're obviously fans of
 <a href="https://github.com/revsys/django-test-plus">
 django-test-plus
 </a>
 , having written it, and have found it very valuable over the last few years at cutting down on all of the boilerplate test code we have to write with our clients.
 
 
 
 
 It even helps sell certain stubborn clients on the idea that testing can be done in a not so painful way!
 
 
 Today we're happy to announce that you can now use django-test-plus as a real pytest fixture. Here's a quick example:
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>def test_some_view(tp):
 tp.get('some-view-name')
 tp.response_200()


def test_some_url_stuff(tp):
 expected_url = '/api/'
 reversed_url = tp.reverse('api')
 assert expected_url == reversed_url
</pre>
 </div>
</div>
<div class="block-content">
 
 And you get all of the other test plus methods as part of that "tp" fixture. This is a new experimental feature, so we'd love to hear how we can make this better and more "pytest like".
 
 
 
 
 To start using all you need to do is install it and pull in "tp" as a fixture, you install it with just:
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>pip install django-test-plus
</pre>
 </div>
</div>

def test_some_view(tp):
    tp.get('some-view-name')
    tp.response_200()


def test_some_url_stuff(tp):
    expected_url = '/api/'
    reversed_url = tp.reverse('api')
    assert expected_url == reversed_url

 
pip install django-test-plus

django-test-plus has long been a useful helper library with Django. Now we've added some simple pytest fixture support to make it even more useful!

pytest support for django-test-plus

With companies moving to microservices, it's becoming common to have a system scattered across multiple repositories. It's frequent to abstract common patterns and code into private repositories that are then included in each service. But using packages from private repos with Python can be tricky. This guide will guide you through the necessary steps.

<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 
 With companies moving to microservices, it's becoming common to have a system scattered across multiple repositories. It's frequent to abstract common patterns and code into private repositories that are then included in each service.
 
 
 But using packages from private repos with Python can be tricky. This guide will walk you through the necessary steps to correctly use private packages with
 <code>
 pip
 </code>
 or
 <code>
 pipenv
 </code>
 . We have also built
 <a href="https://revsys.com/pip-private-packages/">
 a little app
 </a>
 that will generate the right incantations for you to copy and paste.
 
 <h2>
 Figuring out the URL
 </h2>
 
 First, you need to choose which format you want to use: tarball (ie:
 <code>
 .zip
 </code>
 ) or vcs (most likely
 <code>
 git
 </code>
 ). I prefer just downloading the tarball because it doesn't require any additional vcs software. But if use
 <code>
 pipenv
 </code>
 , you'll have to use vcs (see below).
 
 <h3>
 Authentication
 </h3>
 
 In order to have access to the private repo or tarball, you'll need an authentication token. GitHub and other providers allow you to create a private access token for your user, but because we are going to hardcode such token in code, I strongly recommend to create a 'bot' user and use its access token.
 
 <ul>
 <li>
 Github: access token at
 <code>
 https://github.com/settings/tokens
 </code>
 . You can select just the
 <code>
 repo
 </code>
 scopes.
 </li>
 <li>
 Githost: deploy token at
 <code>
 https://&lt;myorg&gt;.githost.io/&lt;org&gt;/&lt;project&gt;/settings/repository
 </code>
 . You can just select
 <code>
 read_repository
 </code>
 scope.
 </li>
 <li>
 Gitlab: deploy token at
 <code>
 https://gitlab.com/profile/&lt;org&gt;/&lt;project&gt;/settings/repository
 </code>
 . You can just select
 <code>
 read_repository
 </code>
 scope.
 </li>
 <li>
 Bitbucket: access token at
 <code>
 https://bitbucket.org/account/user/&lt;botusername&gt;/app-passwords
 </code>
 . You can just select
 <code>
 repositories/read
 </code>
 scope.
 </li>
 </ul>
 <h3>
 Versioning
 </h3>
 
 You will also need to decide which version of your package you want to install. We'll call this the
 <code>
 ref
 </code>
 . You could target a branch, a commit, or a tag/version. I prefer to use version because it gives me a human-readable point of reference.
 
 <h2>
 Building the url
 </h2>
 
 Once you have your access token and your ref, the next step is figuring out a URL at which
 <code>
 pip
 </code>
 can download or clone the code.
 
 <h3>
 Github
 </h3>
 <ul>
 <li>
 Tarball URL will be
 <code>
 https://&lt;access_token&gt;@github.com/&lt;myorg&gt;/&lt;myproject&gt;/archive/&lt;ref&gt;.zip
 </code>
 </li>
 <li>
 VCS Tarball will be
 <code>
 git+https://&lt;access_token&gt;@github.com/&lt;myorg&gt;/&lt;myproject&gt;.git@&lt;ref&gt;
 </code>
 </li>
 </ul>
 <h3>
 Githost
 </h3>
 <ul>
 <li>
 Tarball URL will be
 <code>
 https://&lt;myorg&gt;.githost.io/&lt;myteam&gt;/&lt;myproject&gt;/repository/archive.zip?private_token=&lt;access_token&gt;&amp;ref=&lt;ref&gt;
 </code>
 </li>
 <li>
 VCS Tarball will be
 <code>
 git+https://&lt;token_username&gt;:&lt;deploy_token@&lt;myorg&gt;.githost.io/&lt;myteam&gt;/&lt;myproject&gt;.git@&lt;ref&gt;
 </code>
 </li>
 </ul>
 <h3>
 Gitlab
 </h3>
 <ul>
 <li>
 Tarball URL will be
 <code>
 https://gitlab.com/&lt;myteam&gt;/&lt;myproject&gt;/repository/archive.zip?private_token=&lt;access_token&gt;&amp;ref=&lt;ref&gt;
 </code>
 </li>
 <li>
 VCS Tarball will be
 <code>
 git+https://&lt;token_username&gt;:&lt;deploy_token&gt;@gitlab.com/&lt;myorg&gt;/&lt;myproject&gt;.git@&lt;ref&gt;
 </code>
 </li>
 </ul>
 <h3>
 Bitbucket
 </h3>
 <ul>
 <li>
 Tarball URL will be
 <code>
 https://&lt;botusername&gt;:&lt;access_token&gt;@bitbucket.org/&lt;myorg&gt;/&lt;myproject&gt;/get/&lt;ref&gt;.zip
 </code>
 </li>
 <li>
 VCS Tarball will be
 <code>
 git+https://&lt;botusername&gt;:&lt;access_token&gt;@bitbucket.org/&lt;myorg&gt;/&lt;myproject&gt;.git@&lt;ref&gt;
 </code>
 </li>
 </ul>
 <h2>
 Specifying the dependency
 </h2>
 <h3>
 <code>
 pip
 </code>
 requirements file
 </h3>
 
 Once you have the URL, you can just add it to you requirements files, or use with
 <code>
 pip install
 </code>
 :
 
 <div class="codehilite">
 <pre><code>$ pip install "&lt;URL&gt;" # You'll need the quotes to escape the URL
</code></pre>
 </div>
 <h3>
 <code>
 Pipfile
 </code>
 </h3>
 
 To add the URL to you
 <code>
 Pipfile
 </code>
 , use this syntax:
 
 <div class="codehilite">
 <pre><code>[packages]
revsys-teams = {file = "&lt;URL&gt;"}
</code></pre>
 </div>
 <h3>
 Private dependency of a private package
 </h3>
 
 If you need to specify your private package as a dependency of another package, you'll have to add the URL in your the dependency's
 <code>
 setup.py
 </code>
 . Note that the exact syntax of the specification is a little tricky:
 
 <div class="codehilite">
 <pre><code>import os
from setuptools import setup, find_packages

setup(
 name='upstreamprivatepackage',
 install_requires=[
 # The 'version' is required. Doesn't have to be an actual version of the package.
 # but it must match what's in the `egg` fragment of its entry in `dependency_links`.
 "&lt;myproject&gt;==0.0.1",
 ],
 dependency_links=[
 # there must be a version identifier in the `egg=` fragment and it must match what's
 # in `install_requires`
 '&lt;url&gt;#egg=&lt;myproject&gt;-0.0.1',
 ],
)
</code></pre>
 </div>
 
 When installing the upstream package, you'll also have to tell
 <code>
 pip
 </code>
 to actually parse the entries in
 <code>
 dependency_links
 </code>
 :
 
 <div class="codehilite">
 <pre><code># `pip` has to be run with the `PIP_PROCESS_DEPENDENCY_LINKS` env var or with `--process-dependency-links` option.
# As of v9.0.1, `pip` will print a big red scary deprecation warning, but that's the only way.
PIP_PROCESS_DEPENDENCY_LINKS=1 pip install ...

# or
pip install ... --process-dependency-links
</code></pre>
 </div>
 </div>
</div>

$ pip install "<URL>" # You'll need the quotes to escape the URL

[packages]
revsys-teams = {file = "<URL>"}

import os
from setuptools import setup, find_packages

setup(
 name='upstreamprivatepackage',
 install_requires=[
 # The 'version' is required. Doesn't have to be an actual version of the package.
 # but it must match what's in the `egg` fragment of its entry in `dependency_links`.
 "<myproject>==0.0.1",
 ],
 dependency_links=[
 # there must be a version identifier in the `egg=` fragment and it must match what's
 # in `install_requires`
 '<url>#egg=<myproject>-0.0.1',
 ],
)

# `pip` has to be run with the `PIP_PROCESS_DEPENDENCY_LINKS` env var or with `--process-dependency-links` option.
# As of v9.0.1, `pip` will print a big red scary deprecation warning, but that's the only way.
PIP_PROCESS_DEPENDENCY_LINKS=1 pip install ...

# or
pip install ... --process-dependency-links

Using Private Packages in Python

Lacey didn&#x27;t have the opportunity to work with Docker at her last job. In this tidbit she steps you through getting started with Docker for Django developers.

FROM python:3.6

ENV PYTHONUNBUFFERED 1
ENV DJANGO_ENV dev
ENV DOCKER_CONTAINER 1

COPY ./requirements.txt /code/requirements.txt
RUN pip install -r /code/requirements.txt

COPY . /code/
WORKDIR /code/

EXPOSE 8000

 
FROM python:3.6

 
ENV PYTHONUNBUFFERED 1

 
ENV DJANGO_ENV dev

 
ENV DOCKER_CONTAINER 1

 
COPY ./requirements.txt /code/requirements.txt

 
RUN pip install -r /code/requirements.txt

 
COPY . /code/

 
WORKDIR /code/

 
EXPOSE 8000

 
version: '3'

services:
  db:
    image: postgres:9.6.5
    volumes:
      - postgres_data:/var/lib/postgresql/data/
  web:
    build: .
    command: bash -c "python /code/manage.py migrate --noinput && python /code/manage.py runserver 0.0.0.0:8000"
    volumes:
      - .:/code
    ports:
      - "8000:8000"
    depends_on:
      - db

volumes:
  postgres_data:

 
version: '3'

 
services:

 
db:
    image: postgres:9.6.5
    volumes:
      - postgres_data:/var/lib/postgresql/data/

 
web:
    build: .
    command: bash -c "python /code/manage.py migrate --noinput && python /code/manage.py runserver 0.0.0.0:8000"
    volumes:
      - .:/code
    ports:
      - "8000:8000"
    depends_on:
      - db

 
volumes:
      - .:/code

 
ports:
      - "8000:8000"

 
depends_on:
      - db

 
volumes:
  postgres_data:

 


 
  Finally, Compose has a rule where you have to list your named volumes in a top-level
  
   volumes
  
  key, so we have done that.
 
 
  Save the docker-compose.yml file.
  
 
 
  In Terminal or your console and from the same directory that contains your Dockerfile and docker-compose.yml file, run
  
   docker-compose up
  
  .
  
 
 
  Assuming you have no errors, navigate to
  
   http://localhost:8000/
  
  in a browser and see your blue screen of success once again!
  
 
 
  Ready for your next step? Check out
  
   Docker: Useful Command Line Stuff
  
  next!
 
 
  Resources
 
 
 
 
 
 
  
   
    Get started with Docker
   
   
  
  
   
    Get started with Docker Compose
   
   
  
  
   
    Quickstart: Compose and Django
   
   
  
  
   
    Best practices for writing Dockerfiles
   
  
  
   
    Dockerizing Django, uWSGI and Postgres the Serious Way
   
   , Oliver Eidel
   
  
 
 
  
   Thanks to Frank Wiles and Jeff Triplett for reviewing drafts of this post.

Lacey didn't have the opportunity to work with Docker at her last job. In this tidbit she steps you through getting started with Docker for Django developers.

A Brief Intro to Docker for Djangonauts

Smaller and faster Python 3.6.x images for you to take advantage of.

<div class="block-content">
 
 Turns out there are some optimizations you can do when compiling Python 3.6 and Python 3.7 that give you some significant speed improvements without any real downside.
 
 
 We've paired with with Google's base Debian image that rips out systemd and all it's dependencies, which you weren't going to be using in a Docker container anyway, which yields a MUCH smaller image.
 
 
 How do you use it? Pretty simple just start your Dockerfile with:
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>FROM revolutionsystems/python:3.7.1-wee-optimized-lto
</pre>
 </div>
</div>
<div class="block-content">
 
 And rebuild your image.
 
 
 How much faster is it? Depending on the benchmark in question it's up to 19% faster and nearly 75% smaller than the official Python images.
 
 
 You can get more details on benchmarks and how all this works on the
 <a href="https://hub.docker.com/r/revolutionsystems/python/">
 dockerhub page
 </a>
 . Information on
 <a href="https://github.com/revsys/optimized-python-docker">
 build details
 </a>
 is also available.
 
 
 Enjoy!
 
 
 Need help effectively using Docker with Python or Django? We can help!
 <a href="https://www.revsys.com/contact/">
 Contact us
 </a>
 or
 <a href="mailto:sales@revsys.com">
 send us a quick email
 </a>
 .
 
</div>

FROM revolutionsystems/python:3.7.1-wee-optimized-lto

 


 
  And rebuild your image.
 
 
  How much faster is it? Depending on the benchmark in question it's up to 19% faster and nearly 75% smaller than the official Python images.
 
 
  You can get more details on benchmarks and how all this works on the
  
   dockerhub page
  
  . Information on
  
   build details
  
  is also available.
 
 
  Enjoy!
 
 
  Need help effectively using Docker with Python or Django? We can help!
  
   Contact us
  
  or
  
   send us a quick email
  
  .

Optimized Python

Sometimes you just need to run a command on a remote host with ssh and Python 3 without any real fuss. Here is the simplest way we have found to do that and retrieve the output. 

<div class="block-content">
 
 <a href="http://www.fabfile.org/">
 Fabric
 </a>
 is great, but isn't 100% compatible with Python 3. &nbsp;There is
 <a href="https://github.com/mathiasertl/fabric/">
 Fabric3
 </a>
 , which does work, but sometimes you don't want to have to issue
 
 `fab &lt;something&gt;`
 
 &nbsp;but instead want to issue a command as part of a larger bit of Python. &nbsp;
 
 
 I ran into this situation today where I wanted to issue a few commands over ssh to a remote host as part of a
 <a href="http://click.pocoo.org/6/">
 Click
 </a>
 command I'm building to do some ops automation here at REVSYS.&nbsp;
 
 
 It's probably not perfect in terms of error handling, but it sure is simple! Here is all you need to do:
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>from fabric.api import env
from fabric.operations import run as fabric_run
from fabric.context_managers import settings, hide

def run(user, host, command):
 """ Run a command on the host. Assumes user has SSH keys setup """
 env.user = user
 env.use_ssh_config = True

 with settings(hide('everything'), host_string=host):
 results = fabric_run(command)

 return results
</pre>
 </div>
</div>
<div class="block-content">
 
 You'll need to install the Fabric3 library with a simple:
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>pip install Fabric3
</pre>
 </div>
</div>
<div class="block-content">
 
 And away you go! All this is doing is re-using Fabric3's run() function while silencing all of the normal Fabric style output and forcing the user, host, and to use the calling environment's ssh configuration settings (like your ~/.ssh/config). &nbsp;
 
 
 We could accomplish this more directly with Paramiko or work some subprocess magic to call our ssh, but this was the simplest and fastest way I solved the ssh problem today.
 
 
 Hope you find this little trick useful! Happy Hacking!&nbsp;
 
</div>

from fabric.api import env
from fabric.operations import run as fabric_run
from fabric.context_managers import settings, hide

def run(user, host, command):
    """ Run a command on the host. Assumes user has SSH keys setup """
    env.user = user
    env.use_ssh_config = True

    with settings(hide('everything'), host_string=host):
        results = fabric_run(command)

    return results

 
pip install Fabric3

 


 
  And away you go! All this is doing is re-using Fabric3's run() function while silencing all of the normal Fabric style output and forcing the user, host, and to use the calling environment's ssh configuration settings (like your ~/.ssh/config).  
 
 
  We could accomplish this more directly with Paramiko or work some subprocess magic to call our ssh, but this was the simplest and fastest way I solved the ssh problem today.
 
 
  Hope you find this little trick useful! Happy Hacking!

Sometimes you just need to run a command on a remote host with ssh and Python 3 without any real fuss. Here is the simplest way we have found to do that and retrieve the output.

Python 3 run command over SSH

Python's atexit handler is one of those things people should use more often.  Some developers aren't even aware that it exists and that makes me sad. 

<div class="block-content">
 
 Did you know Python can automatically do things for you when the process exits? The
 <a href="https://docs.python.org/3/library/atexit.html#atexit-example">
 atexit
 </a>
 &nbsp;is one of those handy, but often forgotten about, aspects of Python. &nbsp;I had cause to use it yet again today with a certain process that was being naughty. &nbsp;
 
 
 The idea is simple, atexit&nbsp;instructs Python to run your function just before the process exits. &nbsp;So when would you want to do that? I'm sure there are other situations, but I typically end up using them when:
 
 
 
 <ul>
 <li>
 I want to log both process startup AND exit
 </li>
 <li>
 Alert someone that a process has died, especially cron jobs that are easily forgotten about
 </li>
 </ul>
 
 A quick example to show you the idea:
 
 
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>import atexit


@atexit.register
def closing():
 print("=== Closing down ===")


while True:
 print("whee!")
</pre>
 </div>
</div>
<div class="block-content">
 
 Using the atexit decorator, we wrap a little function that just prints out that we're closing down. Our while loop, er I mean our
 
 whee loop,
 
 will run until we interrupt the process with a Ctrl-C. &nbsp;The output looks like this:
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee^Cwhee!
whee!
Traceback (most recent call last):
 File "example.py", line 10, in &lt;module&gt;
 print("whee!")
KeyboardInterrupt
=== Closing down ===
</pre>
 </div>
</div>
<div class="block-content">
 
 We see our usual KeyboardInterrupt traceback and then our custom closing message.&nbsp;
 
 
 You can also register your atexit functions without the decorator so you can pass args and kwargs to it to be flexible. &nbsp;
 
 
 Seems pretty perfect, doesn't it? It's not all roses. &nbsp;Python only calls these atexit&nbsp;functions in a couple of scenarios, specifically if:
 
 
 
 <ul>
 <li>
 sys.exit() is called
 </li>
 <li>
 The process completes and is exiting normally
 </li>
 </ul>
 
 However, it&nbsp;
 
 does not
 
 &nbsp;call your atexit handles if Python is killed by a signal, when Python itself has a fatal internal error, or if os._exit() is called directly. &nbsp;So it's not useful to detect kills or kill -9s for sure, but in most other usual exit scenarios it can be handy.&nbsp;
 
 
 Hope you find a use for it!&nbsp;
 
 
 
</div>

import atexit


@atexit.register
def closing():
 print("=== Closing down ===")


while True:
 print("whee!")

 
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee^Cwhee!
whee!
Traceback (most recent call last):
 File "example.py", line 10, in <module>
 print("whee!")
KeyboardInterrupt
=== Closing down ===

 


 
 We see our usual KeyboardInterrupt traceback and then our custom closing message. 
 
 
 You can also register your atexit functions without the decorator so you can pass args and kwargs to it to be flexible.  
 
 
 Seems pretty perfect, doesn't it? It's not all roses.  Python only calls these atexit functions in a couple of scenarios, specifically if:
 
 
 
 
 
 sys.exit() is called
 
 
 The process completes and is exiting normally
 
 
 
 However, it 
 
 does not
 
  call your atexit handles if Python is killed by a signal, when Python itself has a fatal internal error, or if os._exit() is called directly.  So it's not useful to detect kills or kill -9s for sure, but in most other usual exit scenarios it can be handy. 
 
 
 Hope you find a use for it!

Python's atexit handler is one of those things people should use more often.  Some developers aren't even aware that it exists and that makes me sad.

Python: atexit

Getting your configuration from the system environment for your Python and/or Django apps is often the best way to provide security and flexibility. envparse makes it easy. 

<div class="block-content">
 
 Making&nbsp;
 <a href="https://12factor.net/">
 12-Factor apps
 </a>
 &nbsp;is all the rage these days and not without good reasons. Using the system environment to get your app's configuration is probably the most flexible way to pass configuration information once you get used to it.
 
 
 One mental barrier I know I had was I didn't want to
 
 
 always
 
 
 &nbsp;have to pass a bunch of variables just to run my code. The ideal setup is:
 
 
 
 <ul>
 <li>
 Load the env vars from a file in the current directory or any directory above
 
 </li>
 <li>
 Have any variables from the actual environment included and give the ability to override them on a per execution basis on the command line
 </li>
 </ul>
 
 Luckily, the
 <a href="https://pypi.python.org/pypi/envparse/0.2.0">
 envparse
 </a>
 &nbsp;library for Python gives us this pretty easily. &nbsp;There are two ways you can use it. &nbsp;The "standard" way or more manual way. &nbsp;Or you can define up front a schema of variables you expect and ensure they are cast to the types you need.&nbsp;
 
 
 Let's explore the standard way first.&nbsp;
 
 
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>from envparse import env

env.read_envfile()
</pre>
 </div>
</div>
<div class="block-content">
 
 That's all you need to do to load up the environment. The library will walk your file system path looking for a .env file, if it finds one it will load it. &nbsp;If it doesn't it will issue a warning you can safely ignore.&nbsp;
 
 
 You're on your own for validation, however. &nbsp;If you go to access a variable that doesn't exist you'll get a KeyError.&nbsp;
 
 
 Let's assume we have the following in a .env&nbsp;in the current directory:
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>AWS_ACCESS_KEY='blahblahblahblah'
AWS_SECRET_KEY='WeAreNotThatDumb'
CONSULTING_LEVEL='Pro'
</pre>
 </div>
</div>
<div class="block-content">
 
 If we then called our script like this:
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>OTHER='bar' CONSULTING_LEVEL='Expert' python envparse-example.py
</pre>
 </div>
</div>
<div class="block-content">
 
 Now let's look at envparse-example.py, what do you think it would output?
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>from envparse import env

env.read_envfile()

print('ACCESS =', env.str('AWS_ACCESS_KEY'))
print('SECRET =', env.str('AWS_SECRET_KEY'))
print('CONSULTING_LEVEL =', env.str('CONSULTING_LEVEL'))
print('OTHER =', env.str('OTHER', default='foo'))
</pre>
 </div>
</div>
<div class="block-content">
 
 You win a cookie or something if you correctly guessed:
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>ACCESS = AKIAJI4KRV67OPXQZS7Q
SECRET = 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
CONSULTING_LEVEL = Expert
OTHER = bar
</pre>
 </div>
</div>
<div class="block-content">
 
 I want to point out two things about this. First, as you probably already understood the fact we have 'CONSULTING_LEVEL' set in both the file and on the command line, the command line version takes precedence.&nbsp;
 
 
 The second thing I want to point out is the last line in the source above. &nbsp;We overrode 'OTHER' on the command line, but if we hadn't it would be set to the default of 'foo'. &nbsp;This helps allow you to set sane defaults in the absence of the variable being set anywhere at all.&nbsp;
 
 <h2>
 envparse&nbsp;using a Schema
 </h2>
 
 Now that we've covered the more manual and direct way of using envparse, let's play with the Schema support real quick.&nbsp;
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>from envparse import Env

env = Env(
 AWS_ACCESS_KEY=str,
 AWS_SECRET_KEY=str,
 CONSULTING_LEVEL=str,
 OTHER=dict(cast=str, default='foo')

)
env.read_envfile()

print('ACCESS =', env.str('AWS_ACCESS_KEY'))
print('SECRET =', env.str('AWS_SECRET_KEY'))
print('CONSULTING_LEVEL =', env.str('CONSULTING_LEVEL'))
print('OTHER =', env.str('OTHER'))
</pre>
 </div>
</div>
<div class="block-content">
 
 Assuming the same .env file and overrides on the command line this will produce the exact same results. The different here is we import 'Env' (note the capitalization) and set the variables we expect to have.&nbsp;
 
 
 If something is missing from our environment, envparse&nbsp;raises a envparse.ConfigurationError rather than getting the more generic KeyError when we go to access to a missing value.
 
 
 Using a schema like this also gives us options for settings defaults as part of the definition rather than when calling for the value itself. &nbsp;envparse has a few other useful options for casting more complex types and giving you the ability to pre or post process the values when they are read which can be useful to help normalize values. &nbsp;
 
 <h2>
 Using envparse with Click
 </h2>
 
 I'm a huge fan of
 <a href="http://click.pocoo.org/6/">
 Click
 </a>
 &nbsp;and while it has great support for
 <a href="http://click.pocoo.org/6/options/#values-from-environment-variables">
 pulling values out of the environment for you
 </a>
 , it doesn't support using a .env&nbsp;file directly. &nbsp;Lucky for us it's straightforward to use envparse with Click. If you need options that can come from a .env file, the environment directly, or passed on the command line you can do this:
 
 
 
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>import click

from envparse import Env

env = Env(
 AWS_ACCESS_KEY=str,
 AWS_SECRET_KEY=str,
 CONSULTING_LEVEL=str,
 OTHER=dict(cast=str, default='foo')

)
env.read_envfile()

@click.command()
@click.option('--access-key', default=env.str('AWS_ACCESS_KEY'))
@click.option('--secret-key', default=env.str('AWS_SECRET_KEY'))
def testing(access_key, secret_key):
 print(access_key)
 print(secret_key)

if __name__ == '__main__':
 testing()
</pre>
 </div>
</div>
<div class="block-content">
 
 The trick here is to use envparse to read in your expected values, whether they come from a .env file or are passed on in via the environment it doesn't matter. &nbsp;We then use the value from envparse to set the default for Click. &nbsp;If we need to, we can still override this on the command line and everyone is happy.
 
 
 Hope this helps you make your Python app's configuration more flexible!&nbsp;
 
</div>

from envparse import env

env.read_envfile()

 
AWS_ACCESS_KEY='blahblahblahblah'
AWS_SECRET_KEY='WeAreNotThatDumb'
CONSULTING_LEVEL='Pro'

 
OTHER='bar' CONSULTING_LEVEL='Expert' python envparse-example.py

 
from envparse import env

env.read_envfile()

print('ACCESS =', env.str('AWS_ACCESS_KEY'))
print('SECRET =', env.str('AWS_SECRET_KEY'))
print('CONSULTING_LEVEL =', env.str('CONSULTING_LEVEL'))
print('OTHER =', env.str('OTHER', default='foo'))

 
ACCESS = AKIAJI4KRV67OPXQZS7Q
SECRET = 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
CONSULTING_LEVEL = Expert
OTHER = bar

 
from envparse import Env

env = Env(
    AWS_ACCESS_KEY=str,
    AWS_SECRET_KEY=str,
    CONSULTING_LEVEL=str,
    OTHER=dict(cast=str, default='foo')

)
env.read_envfile()

print('ACCESS =', env.str('AWS_ACCESS_KEY'))
print('SECRET =', env.str('AWS_SECRET_KEY'))
print('CONSULTING_LEVEL =', env.str('CONSULTING_LEVEL'))
print('OTHER =', env.str('OTHER'))

 
import click

from envparse import Env

env = Env(
    AWS_ACCESS_KEY=str,
    AWS_SECRET_KEY=str,
    CONSULTING_LEVEL=str,
    OTHER=dict(cast=str, default='foo')

)
env.read_envfile()

@click.command()
@click.option('--access-key', default=env.str('AWS_ACCESS_KEY'))
@click.option('--secret-key', default=env.str('AWS_SECRET_KEY'))
def testing(access_key, secret_key):
    print(access_key)
    print(secret_key)

if __name__ == '__main__':
    testing()

 


 
  The trick here is to use envparse to read in your expected values, whether they come from a .env file or are passed on in via the environment it doesn't matter.  We then use the value from envparse to set the default for Click.  If we need to, we can still override this on the command line and everyone is happy.
 
 
  Hope this helps you make your Python app's configuration more flexible!

Getting your configuration from the system environment for your Python and/or Django apps is often the best way to provide security and flexibility. envparse makes it easy.

Python 12-factor apps with envparse

Have a URL but can't figure out what view it is supposed to use? Use resolve() to easily and quickly determine it just as Django does.

<div class="block-content">
 
 One common practice in Django is reversing URL:
 
 
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>from django.urls import reverse

reverse('my_url_name', kwargs={'pk': 1})
</pre>
 </div>
</div>
<div class="block-content">
 
 But very often, in large projects, it's hard to remember exactly what the name of the URL is. Furthermore, you maybe using 3rd party libraries that include URLs you want to refer to.
 
 
 If you know what the URL looks like, but you can't remember the name or its arguments, a quick and easy way to find them out is use the `resolve` function:
 
 
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>from django.urls import resolve

resolve('/myurl/2')
# ResolverMatch(func=myapp.views.MyModelViewSet, args=(), kwargs={u'pk': u'2'}, url_name=mymodel-detail, app_name=None, namespaces=['api'])
</pre>
 </div>
</div>
<div class="block-content">
 
 As you can see, the output includes all you need to&nbsp;reverse&nbsp;the URL, including arguments,&nbsp;app_name&nbsp;and&nbsp;namespaces.
 
</div>
<div class="block-html">
 <div class="block-content">
 <div class="rich-text">
 <h6>
 UPDATE
 </h6>
 
 The
 <code>
 django-extensions
 </code>
 package provides another easy way to find out which view. After installing it, you can simply use its
 <code>
 show_urls
 </code>
 and
 <code>
 grep
 </code>
 ping for the URL you're looking for:
 
 </div>
 </div>
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>$ ./manage.py show_urls | grep "/myurl/&lt;pk&gt;"
</pre>
 </div>
</div>
<div class="block-content">
 
 Thanks to
 
 @bmihelac
 
 &nbsp;for the suggestion!
 
 
 
 
</div>

from django.urls import reverse

reverse('my_url_name', kwargs={'pk': 1})

 
from django.urls import resolve

resolve('/myurl/2')
# ResolverMatch(func=myapp.views.MyModelViewSet, args=(), kwargs={u'pk': u'2'}, url_name=mymodel-detail, app_name=None, namespaces=['api'])

 
$ ./manage.py show_urls | grep "/myurl/<pk>"

 


 
 Thanks to
 
 @bmihelac
 
  for the suggestion!

reverse()'s soulmate resolve()

You can use a variable to set the name of the template you are extending when using Django templates.

<div class="block-content">
 
 One of my all favorite template tricks goes back to my LJ World days.
 
 
 Most people don't realize that the&nbsp;
 <a href="https://docs.djangoproject.com/en/1.9/ref/templates/builtins/#extends">
 extends
 </a>
 &nbsp;tag accepts a variable and not just a hard-coded literal value. This allows for magical things to happen when combined with a default.
 
 
 
 The following block extends a base template named "base.html" if the&nbsp;
 
 base_template
 
 &nbsp;is not set.
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>{% extends base_template|default:"base.html" %}
</pre>
 </div>
</div>
<div class="block-content">
 
 For any page that you want to programmatically override the base template, you may do so by adding a&nbsp;base_template context variable to your views context values which are exposed to your template.
 
</div>
<div class="block-code">
 <div class="codehilite">
 <pre>from django.views.generic import TemplateView


class MarketingView(TemplateView):

 template_name = 'marketing_landing.html'

 def get_context_data(self, **kwargs):
 context = super(MarketingView, self).get_context_data(**kwargs)
 context['base_template'] = 'base_marketing.html'
 return context
</pre>
 </div>
</div>
<div class="block-content">
 
 This simple trick opens the door to do fun things with your website based on the day of the year or something useful like A/B test two different versions of your website's landing page. The possibilities are endless.
 
</div>

{% extends base_template|default:"base.html" %}

 
from django.views.generic import TemplateView


class MarketingView(TemplateView):

    template_name = 'marketing_landing.html'

    def get_context_data(self, **kwargs):
        context = super(MarketingView, self).get_context_data(**kwargs)
        context['base_template'] = 'base_marketing.html'
        return context

 


 
  This simple trick opens the door to do fun things with your website based on the day of the year or something useful like A/B test two different versions of your website's landing page. The possibilities are endless.

Extend Django Templates Smartly

Our recap of the Django&#x27;s 10th Birthday Party

<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 <h2>
 Happy 10th Birthday Django!
 </h2>
 
 <img alt="" src="http://photos.frankwiles.com/Events/Django-Birthday/i-pRMN7Sr/0/M/150711jasondailey841-M.jpg"/>
 
 
 Whew, what a conference! The talks were great, the venue was comfortable, and the BBQ was amazing. But that was only part of it. Many of the talks focused on the Django community and you could feel it in the room. We laughed from our bellies and teared up each time someone mentioned
 <a href="https://www.djangoproject.com/weblog/2013/sep/16/announcing-malcolm-tredinnick-memorial-prize/">
 Malcolm
 </a>
 .
 
 
 I had the honor of giving the closing address, which was really just a thinly veiled way for me to have a Steve Jobs "oh and one more thing" moment. We had secretly conspired with The World Company, the place where Django was born, to open up the building so all of the attendees could get a quick tour and see
 
 EXACTLY
 
 where Django originated. Liberty Hall, where we held the conference, is conveniently across the alley from the Journal-World building so we all stood up, grabbed our bags, and walked over. The photo above is most of us in front of the News Center building where Django was born.
 
 
 The night before the conference we had a very loosely planned evening of dinner and drinks which also included a quick Skype session with the Django Girls group in Australia to kick off the celebration. It was a great relaxed way to catch up with old friends.
 
 <h2>
 Videos
 </h2>
 
 The
 <a href="http://pyvideo.org/category/71/django-birthday">
 videos are up on PyVideo.org
 </a>
 for you all to enjoy. As always
 <a href="http://nextdayvideo.com/">
 Next Day Video
 </a>
 had them up so fast it was a constant stream of "OMG my talk is already up" Tweets.
 
 <h2>
 Photos
 </h2>
 
 Even before Adrian mentioned in his talk about taking more photos of your projects for nostalgia purposes, we had it in our plans for Django Birthday. There is no telling if Django will make it to 2025, but assuming it does we have plenty of ammo for the next birthday party. I took a bunch of photos and we hired
 <a href="http://www.daileyimages.com/">
 Dailey Images
 </a>
 to ensure we captured all of the right moments.
 
 
 Some initial quickly edited shots from Jason Dailey, the photos I took, and some from my friend Joe Griffin are in my
 <a href="http://photos.frankwiles.com/Events/Django-Birthday/">
 Django Birthday
 </a>
 photo album.
 
 
 Kenneth Love also setup an
 <a href="https://www.icloud.com/sharedalbum/#B04GWZuqDGv43mF">
 iCloud album
 </a>
 of some of his shots. Including one of the newspaper the next day with a front page photo of the block party.
 
 <h2>
 The Party
 </h2>
 
 <img alt="" src="http://photos.frankwiles.com/Events/Django-Birthday/i-nTMNj9g/0/X3/Django_July11_2015-9-X3.jpg"/>
 
 
 Eldarion and RevSys sponsored the party with Lincoln Loop sponsoring the birthday cake.
 
 
 While it was a bit warmer than I would like, the party was epic. We had all of the things you would expect at a 10 year old geeky kids birthday party. Snow cones, balloon animals, Gizmo the robot wandering around, and of course a bouncy castle. The Batmobile even made an appearance, but that wasn't planned... it's just
 <a href="https://www.google.com/webhp?sourceid=chrome-instant&amp;ion=1&amp;espv=2&amp;es_th=1&amp;ie=UTF-8#q=%23LFK&amp;es_th=1">
 #LFK
 </a>
 
 
 For the more adult of us, we also had 3 musical performances and started the night off with Adrian himself playing Django Reinhardt. Yep that's right, the guy who created Django, playing Django, at Django's 10th birthday.
 
 
 Our friends at
 <a href="http://www.thesandbar.com">
 The SandBar
 </a>
 were awesome enough to help us throw this block party and showed everyone an amazing time. We had the usual things you would expect, but also a special Python/Django themed craft cocktail tent and this awesome shrimp &amp; rice in a pineapple food option that I have to try to recreate at home now.
 
 
 When we started planning this party Jacob made me promise to not have any ponies, but I couldn't resist one. So in addition to our big cakes for all attendees we had the baker create a small cutting cake and let Adrian have the honor of cutting the pony in half.
 
 
 I want to personally thank everyone who made the trip out to our little weird bit of the world. I hope you enjoyed yourselves and come back again very soon. Things went so well, I'm not sure I can wait until 2025. We may have to celebrate Django's Sweet 16 or something in the mean time.
 
 <h2>
 Many thanks to our great sponsors for helping us put on a great and fun event!
 </h2>
 <ul>
 <li>
 <a href="http://eldarion.com/">
 Eldarion
 </a>
 </li>
 <li>
 <a href="https://p.ota.to/">
 Potato
 </a>
 </li>
 <li>
 <a href="http://plushrugs.com/">
 PlushRugs.com
 </a>
 </li>
 <li>
 <a href="https://lincolnloop.com/">
 Lincoln Loop
 </a>
 </li>
 <li>
 <a href="https://reelio.com/">
 Reelio
 </a>
 </li>
 <li>
 <a href="https://teamtreehouse.com/">
 Treehouse
 </a>
 </li>
 </ul>
 
 And our travel sponsors:
 
 <ul>
 <li>
 <a href="https://www.python.org/psf/">
 The Python Software Foundation
 </a>
 </li>
 <li>
 <a href="https://www.djangoproject.com/">
 The Django Software Foundation
 </a>
 </li>
 </ul>
 </div>
</div>

Happy 10th Birthday Django!
  
  
   
  
  
   Whew, what a conference! The talks were great, the venue was comfortable, and the BBQ was amazing. But that was only part of it. Many of the talks focused on the Django community and you could feel it in the room. We laughed from our bellies and teared up each time someone mentioned
   
    Malcolm
   
   .
  
  
   I had the honor of giving the closing address, which was really just a thinly veiled way for me to have a Steve Jobs "oh and one more thing" moment. We had secretly conspired with The World Company, the place where Django was born, to open up the building so all of the attendees could get a quick tour and see
   
    EXACTLY
   
   where Django originated. Liberty Hall, where we held the conference, is conveniently across the alley from the Journal-World building so we all stood up, grabbed our bags, and walked over. The photo above is most of us in front of the News Center building where Django was born.
  
  
   The night before the conference we had a very loosely planned evening of dinner and drinks which also included a quick Skype session with the Django Girls group in Australia to kick off the celebration. It was a great relaxed way to catch up with old friends.
  
  
   Videos
  
  
   The
   
    videos are up on PyVideo.org
   
   for you all to enjoy. As always
   
    Next Day Video
   
   had them up so fast it was a constant stream of "OMG my talk is already up" Tweets.
  
  
   Photos
  
  
   Even before Adrian mentioned in his talk about taking more photos of your projects for nostalgia purposes, we had it in our plans for Django Birthday. There is no telling if Django will make it to 2025, but assuming it does we have plenty of ammo for the next birthday party. I took a bunch of photos and we hired
   
    Dailey Images
   
   to ensure we captured all of the right moments.
  
  
   Some initial quickly edited shots from Jason Dailey, the photos I took, and some from my friend Joe Griffin are in my
   
    Django Birthday
   
   photo album.
  
  
   Kenneth Love also setup an
   
    iCloud album
   
   of some of his shots. Including one of the newspaper the next day with a front page photo of the block party.
  
  
   The Party
  
  
   
  
  
   Eldarion and RevSys sponsored the party with Lincoln Loop sponsoring the birthday cake.
  
  
   While it was a bit warmer than I would like, the party was epic. We had all of the things you would expect at a 10 year old geeky kids birthday party. Snow cones, balloon animals, Gizmo the robot wandering around, and of course a bouncy castle. The Batmobile even made an appearance, but that wasn't planned... it's just
   
    #LFK
   
  
  
   For the more adult of us, we also had 3 musical performances and started the night off with Adrian himself playing Django Reinhardt. Yep that's right, the guy who created Django, playing Django, at Django's 10th birthday.
  
  
   Our friends at
   
    The SandBar
   
   were awesome enough to help us throw this block party and showed everyone an amazing time. We had the usual things you would expect, but also a special Python/Django themed craft cocktail tent and this awesome shrimp & rice in a pineapple food option that I have to try to recreate at home now.
  
  
   When we started planning this party Jacob made me promise to not have any ponies, but I couldn't resist one. So in addition to our big cakes for all attendees we had the baker create a small cutting cake and let Adrian have the honor of cutting the pony in half.
  
  
   I want to personally thank everyone who made the trip out to our little weird bit of the world. I hope you enjoyed yourselves and come back again very soon. Things went so well, I'm not sure I can wait until 2025. We may have to celebrate Django's Sweet 16 or something in the mean time.
  
  
   Many thanks to our great sponsors for helping us put on a great and fun event!
  
  
   
    
     Eldarion
    
   
   
    
     Potato
    
   
   
    
     PlushRugs.com
    
   
   
    
     Lincoln Loop
    
   
   
    
     Reelio
    
   
   
    
     Treehouse
    
   
  
  
   And our travel sponsors:
  
  
   
    
     The Python Software Foundation
    
   
   
    
     The Django Software Foundation

Our recap of the Django's 10th Birthday Party

Django Birthday: Recap

Django is turning 10 years old this summer and we&#x27;re throwing it a birthday party in the city where it was born.

<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 
 So as you may know, Django is turning 10 years old this summer and we&rsquo;re throwing it a
 <a href="https://djangobirthday.com">
 birthday party
 </a>
 . It&rsquo;s a one day tech conference and one day of sprints mere feet from where Adrian, Simon, Jacob, and Wilson toiled away on &ldquo;
 
 the CMS
 
 &rdquo;, as it was called then. That code became the first of a few amazing refreshes of
 <a href="http://www.lawrence.com">
 Lawrence.com
 </a>
 and the
 
 very first
 
 Django website.
 
 
 We&rsquo;ve got an amazing line up of
 <a href="https://djangobirthday.com/speakers/">
 speakers
 </a>
 of course. And we want to celebrate Django obviously, but secretly we have another small motivation. See we love Lawrence, despite the fact it&rsquo;s surrounded by Kansas, and we&rsquo;ve grown tired of answering questions along the lines of &ldquo;Ummm why are you in Kansas?&rdquo; So we&rsquo;re going to expose our Django community friends to as much as this area has to offer to help explain why we choose to live here. And who knows maybe one of them will fall in love, as we have, and build their next startup here.
 
 <h2>
 So what can you expect?
 </h2>
 
 Most everyone agrees the best part of Lawrence is our downtown, so we&rsquo;re having the event downtown and encouraging everyone to stay in or near the area. This makes everything special in Lawrence within walking distance. There&rsquo;s great local restaurants, quirky stores, and being a college town lots of interesting places to grab a libation should you so choose.
 
 
 We&rsquo;re having the event at
 <a href="https://www.google.com/search?q=liberty+hall+lawrence&amp;espv=2&amp;biw=1439&amp;bih=1010&amp;source=lnms&amp;tbm=isch&amp;sa=X&amp;ei=MfOBVZLRFpSZoQTFhoDIDw&amp;ved=0CAcQ_AUoAg">
 Liberty Hall
 </a>
 which has acted as a community-meeting house and venue since 1856! If you&rsquo;re interested you can
 <a href="http://www.libertyhall.net/about/history">
 read more about its history
 </a>
 . Lawrence is weird and funky, which is reflected in the wide variety of entertainers that have performed there. Some highlights include: Oscar Wilde, Tina Turner, Wu Tang Clan, Henry Rollins, Tori Amos, The Civil Wars, and for its 100th anniversary party
 <a href="http://www.pitch.com/FastPitch/archives/2012/06/22/the-flaming-lips-circus-last-night-at-liberty-hall">
 The Flaming Lips
 </a>
 which I&rsquo;m quite sad to have missed. Lawrence has a long and great music history, but much of that happened in the 90s at
 <a href="https://en.wikipedia.org/wiki/Outhouse_(venue)">
 The Outhouse
 </a>
 (which is now sadly a strip club so we
 
 won&rsquo;t
 
 be going there) and another venue down the street named the
 <a href="http://thebottlenecklive.com/about/">
 Bottleneck
 </a>
 .
 
 
 Kansas is known for its BBQ, so of course we&rsquo;re having Kansas City&rsquo;s best
 <a href="http://www.joeskc.com/about/">
 Oklahoma Joe&rsquo;s
 </a>
 cater lunch. In 2009, Anthony Bourdain listed Joe&rsquo;s in his &ldquo;Thirteen Places to Eat Before You Die&rdquo;. Yeah it&rsquo;s that good.
 
 <h2>
 But it&rsquo;s a party right?
 </h2>
 
 Oh yes it&rsquo;s a party all right! We&rsquo;ve gotten special permission from the City of Lawrence to close down a block of downtown and hold a block party! We&rsquo;ll have a few live bands, including Adrian, food, beer, a special cocktail tent for our more refined guests, snow cones, and some other fun family friendly things for the kids.
 
 
 See many in Lawrence don&rsquo;t know about Django and we want to change that. By having a block party all of Lawrence can come down and help celebrate, instead of having it just be a small gathering of us geeks.
 
 
 Oh and there will be cake of course&hellip; it&rsquo;s not a birthday party without cake.
 
 <h2>
 Many thanks to our great sponsors for helping us put on a great and fun event!
 </h2>
 <ul>
 <li>
 <a href="https://p.ota.to/">
 Potato
 </a>
 </li>
 <li>
 <a href="http://plushrugs.com/">
 PlushRugs.com
 </a>
 </li>
 <li>
 <a href="https://lincolnloop.com/">
 Lincoln Loop
 </a>
 </li>
 <li>
 <a href="https://reelio.com/">
 Reelio
 </a>
 </li>
 <li>
 <a href="https://teamtreehouse.com/">
 Treehouse
 </a>
 </li>
 </ul>
 
 And our travel sponsors:
 
 <ul>
 <li>
 <a href="https://www.python.org/psf/">
 The Python Software Foundation
 </a>
 </li>
 <li>
 <a href="https://www.djangoproject.com/">
 The Django Software Foundation
 </a>
 </li>
 </ul>
 </div>
</div>

So as you may know, Django is turning 10 years old this summer and we’re throwing it a
   
    birthday party
   
   . It’s a one day tech conference and one day of sprints mere feet from where Adrian, Simon, Jacob, and Wilson toiled away on “
   
    the CMS
   
   ”, as it was called then. That code became the first of a few amazing refreshes of
   
    Lawrence.com
   
   and the
   
    very first
   
   Django website.
  
  
   We’ve got an amazing line up of
   
    speakers
   
   of course. And we want to celebrate Django obviously, but secretly we have another small motivation. See we love Lawrence, despite the fact it’s surrounded by Kansas, and we’ve grown tired of answering questions along the lines of “Ummm why are you in Kansas?” So we’re going to expose our Django community friends to as much as this area has to offer to help explain why we choose to live here. And who knows maybe one of them will fall in love, as we have, and build their next startup here.
  
  
   So what can you expect?
  
  
   Most everyone agrees the best part of Lawrence is our downtown, so we’re having the event downtown and encouraging everyone to stay in or near the area. This makes everything special in Lawrence within walking distance. There’s great local restaurants, quirky stores, and being a college town lots of interesting places to grab a libation should you so choose.
  
  
   We’re having the event at
   
    Liberty Hall
   
   which has acted as a community-meeting house and venue since 1856! If you’re interested you can
   
    read more about its history
   
   . Lawrence is weird and funky, which is reflected in the wide variety of entertainers that have performed there. Some highlights include: Oscar Wilde, Tina Turner, Wu Tang Clan, Henry Rollins, Tori Amos, The Civil Wars, and for its 100th anniversary party
   
    The Flaming Lips
   
   which I’m quite sad to have missed. Lawrence has a long and great music history, but much of that happened in the 90s at
   
    The Outhouse
   
   (which is now sadly a strip club so we
   
    won’t
   
   be going there) and another venue down the street named the
   
    Bottleneck
   
   .
  
  
   Kansas is known for its BBQ, so of course we’re having Kansas City’s best
   
    Oklahoma Joe’s
   
   cater lunch. In 2009, Anthony Bourdain listed Joe’s in his “Thirteen Places to Eat Before You Die”. Yeah it’s that good.
  
  
   But it’s a party right?
  
  
   Oh yes it’s a party all right! We’ve gotten special permission from the City of Lawrence to close down a block of downtown and hold a block party! We’ll have a few live bands, including Adrian, food, beer, a special cocktail tent for our more refined guests, snow cones, and some other fun family friendly things for the kids.
  
  
   See many in Lawrence don’t know about Django and we want to change that. By having a block party all of Lawrence can come down and help celebrate, instead of having it just be a small gathering of us geeks.
  
  
   Oh and there will be cake of course… it’s not a birthday party without cake.
  
  
   Many thanks to our great sponsors for helping us put on a great and fun event!
  
  
   
    
     Potato
    
   
   
    
     PlushRugs.com
    
   
   
    
     Lincoln Loop
    
   
   
    
     Reelio
    
   
   
    
     Treehouse
    
   
  
  
   And our travel sponsors:
  
  
   
    
     The Python Software Foundation
    
   
   
    
     The Django Software Foundation

Django is turning 10 years old this summer and we're throwing it a birthday party in the city where it was born.

Django Birthday Party

Improve your Django Tests using django-test-plus&#x27; helpful additions to the standard Django TestCase

<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 
 You write tests. Or at least you should be writing tests! As Jacob always says,
 
 "Code without tests is broken as designed."
 
 
 
 Unfortunately you have to write a ton of boilerplate code to test even the simplest of things. This isn't true of just Django, but most languages and frameworks. There are lots of tools like
 <a href="https://factoryboy.readthedocs.org/en/latest/">
 Factory Boy
 </a>
 and
 <a href="http://pytest.org/">
 pytest
 </a>
 that help, but we can do better.
 
 
 If you're looking for how to get started writing Django tests I'd suggest reading the
 <a href="https://docs.djangoproject.com/en/dev/topics/testing/">
 excellent testing documentation
 </a>
 first and also consider picking up Harry Percival's excellent book
 <a href="http://www.amazon.com/gp/product/1449364829/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=390957&amp;creativeASIN=1449364829&amp;linkCode=as2&amp;tag=revosystblog-20&amp;linkId=VG3GUGRNAQXLZRVR">
 Test-Driven Development with Python
 </a>
 <img alt="" src="http://ir-na.amazon-adsystem.com/e/ir?t=revosystblog-20&amp;l=as2&amp;o=1&amp;a=1449364829"/>
 
 
 After the 400th time of copying over the same base test methods into various personal and client projects, I finally got around to packaging up and releasing all of our useful test additions into a module
 <a href="https://pypi.python.org/pypi/django-test-plus">
 django-test-plus
 </a>
 . I confess, I did this so I didn't have to copy it around anymore as much as wanting to share it with the world. My laziness is your gain. Come be lazy with me!
 
 <h2>
 Let's have some testing fun
 </h2>
 
 I think this module is best explained by examples. So let's just dive right in. First off you need to install django-test-plus with:
 
 <div class="codehilite">
 <pre><code>pip install django-test-plus
</code></pre>
 </div>
 <h3>
 Then instead of this...
 </h3>
 <div class="codehilite">
 <pre><code>from django.test import TestCase
from django.core.urlresolvers import reverse

class MyViewTest(TestCase):

 def test_some_view(self):
 url = reverse('my-url-name')
 response = self.client.get(url)
 self.assertEqual(response.status_code, 200)
</code></pre>
 </div>
 <h3>
 Do this...
 </h3>
 <div class="codehilite">
 <pre><code>from test_plus.test import TestCase

class MyViewTest(TestCase):

 def test_some_view(self):
 response = self.get('my-url-name')
 self.response_200(response)
</code></pre>
 </div>
 
 For the sake of brevity, I'll be excluding the imports and test class in the rest of these examples and just focus on the methods themselves.
 
 <h3>
 Or even this...
 </h3>
 
 Getting a named URL and checking that status is 200 is such a common pattern, it has it's own helper. This will get the named URL, check that the status is 200 and return the response for other checks.
 
 <div class="codehilite">
 <pre><code>def test_some_view(self):
 response = self.get_check_200('my-url-name')
</code></pre>
 </div>
 <h3>
 Instead of...
 </h3>
 <div class="codehilite">
 <pre><code>def test_with_reverse_args(self):
 url = reverse('some-url', kwargs={
 'year': 2015, 
 'slug': 'my-this-is-getting-long'
 })
 response = self.client.get(url)
</code></pre>
 </div>
 <h3>
 Do this...
 </h3>
 <div class="codehilite">
 <pre><code>def test_with_reverse_args(self):
 response = self.get('some-url', year=2015, slug='much-better')
</code></pre>
 </div>
 <h2>
 HTTP Posts work the same way
 </h2>
 <h3>
 Instead of...
 </h3>
 <div class="codehilite">
 <pre><code>def post_with_reverse_args(self):
 url = reverse('some-url', kwargs={
 'year': 2015, 
 'slug': 'my-this-is-getting-long'
 })
 data = {'body': 'long-way'}
 response = self.client.post(url. data=data)
</code></pre>
 </div>
 
 `
 
 <h3>
 Save your keystrokes for another day with...
 </h3>
 <div class="codehilite">
 <pre><code>def test_post_better(self):
 response = self.post('some-url', 
 year=2015, 
 slug='much-better', 
 data={'body': 'lazy-way'}
 )
</code></pre>
 </div>
 <h3>
 Sometimes you still need reverse...
 </h3>
 
 So it's included for you. No need to import it yourself.
 
 <div class="codehilite">
 <pre><code>def test_reversing(self):
 response = self.get('my-named-url')
 test_url = self.reverse('some-other-url', pk=12)
 self.assertEqual(response.context['next'], test_url)
</code></pre>
 </div>
 
 That's better, but that's still a bit rough on the hands. Let's make it a touch better with this version which uses our
 <code>
 get_context()
 </code>
 method:
 
 <div class="codehilite">
 <pre><code>def test_reversing(self):
 response = self.get('my-named-url')
 test_url = self.reverse('some-other-url', pk=12)
 self.assertEqual(self.get_context('next'), test_url)
</code></pre>
 </div>
 
 We keep the last response received in
 <code>
 self.last_response
 </code>
 for you so there isn't a reason to have to pass it around all over the place.
 
 
 Often you need to test the values of several context variables, so let's make that a bit easier.
 
 <div class="codehilite">
 <pre><code>def test_several_values(self):
 self.get('my-view')
 self.assertContext('key1', 'value1')
 self.assertContext('key2', 'value2')
 self.assertContext('key3', False)
</code></pre>
 </div>
 <h2>
 What about other statuses?
 </h2>
 
 Don't worry we've got you covered there are several
 <code>
 response_XXX()
 </code>
 methods to test for other common status code, for example:
 
 <div class="codehilite">
 <pre><code>def test_not_found(self):
 response = self.get('no-there')
 self.response_404(respones)
</code></pre>
 </div>
 <h2>
 Authentication and Users
 </h2>
 
 When testing out Django views you often need to make some users, login in as them, and poke around. Let's make that easier too!
 
 <div class="codehilite">
 <pre><code>def test_needs_login(self):
 # Make a user 
 user = self.make_user('testuser')

 # Make sure we protected the view
 self.assertLoginRequired('my-protected-view')

 with self.login(user):
 self.get('my-protected-view')
 self.assertContext('secret', True)
</code></pre>
 </div>
 <h3>
 Performance
 </h3>
 
 It's easy to make a few template changes that seem inconsequential only to have your database query count blow up to the size of Warren Buffett's checking account. Django provides the
 <code>
 assertNumQueries
 </code>
 assertion context to check the query count, but that is a static count. Often the results are slightly variable, so django-test-plus as
 <code>
 assertNumQueriesLessThan
 </code>
 .
 
 <div class="codehilite">
 <pre><code>def test_does_not_get_crazy(self):
 with self.assertNumQueriesLessThan(25):
 variable_query_count_function()
</code></pre>
 </div>
 <h3>
 Ultimate Lazy
 </h3>
 
 Maybe it's a toy project or you're adding tests to a project that previously didn't have any. Some tests are better than no tests right? So we provide a quick view check with
 <code>
 assertGoodView
 </code>
 :
 
 <div class="codehilite">
 <pre><code>def test_better_than_nothing(self):
 self.assertGoodView('first-view')
 self.assertGoodView('second-view')
 self.assertGoodView('something-else')
</code></pre>
 </div>
 
 What's this do? It gets the view at the named URL, ensures the status code is 200 and tests that the number of queries run is less than 50. Oh and it returns the response if you want it for other purposes. Not amazing, but better than having no test coverage.
 
 <h2>
 The future
 </h2>
 
 I've been using various versions of this module for years, but there is always room for improvement. Happy to take pull requests submissions for new methods that will be generally useful. Or maybe one day I'll be less lazy and see about merging these into Django core.
 
 
 Hope you like the module and find it useful. Happy Testing!
 
 </div>
</div>

pip install django-test-plus

from django.test import TestCase
from django.core.urlresolvers import reverse

class MyViewTest(TestCase):

    def test_some_view(self):
        url = reverse('my-url-name')
        response = self.client.get(url)
        self.assertEqual(response.status_code, 200)

from test_plus.test import TestCase

class MyViewTest(TestCase):

    def test_some_view(self):
        response = self.get('my-url-name')
        self.response_200(response)

def test_some_view(self):
    response = self.get_check_200('my-url-name')

def test_with_reverse_args(self):
    url = reverse('some-url', kwargs={
        'year': 2015, 
        'slug': 'my-this-is-getting-long'
    })
    response = self.client.get(url)

def test_with_reverse_args(self):
    response = self.get('some-url', year=2015, slug='much-better')

def post_with_reverse_args(self):
    url = reverse('some-url', kwargs={
        'year': 2015, 
        'slug': 'my-this-is-getting-long'
    })
    data = {'body': 'long-way'}
    response = self.client.post(url. data=data)

def test_post_better(self):
    response = self.post('some-url', 
        year=2015, 
        slug='much-better', 
        data={'body': 'lazy-way'}
    )

def test_reversing(self):
    response = self.get('my-named-url')
    test_url = self.reverse('some-other-url', pk=12)
    self.assertEqual(response.context['next'], test_url)

def test_reversing(self):
    response = self.get('my-named-url')
    test_url = self.reverse('some-other-url', pk=12)
    self.assertEqual(self.get_context('next'), test_url)

def test_several_values(self):
    self.get('my-view')
    self.assertContext('key1', 'value1')
    self.assertContext('key2', 'value2')
    self.assertContext('key3', False)

def test_not_found(self):
    response = self.get('no-there')
    self.response_404(respones)

def test_needs_login(self):
    # Make a user 
    user = self.make_user('testuser')

    # Make sure we protected the view
    self.assertLoginRequired('my-protected-view')

    with self.login(user):
          self.get('my-protected-view')
          self.assertContext('secret', True)

def test_does_not_get_crazy(self):
    with self.assertNumQueriesLessThan(25):
        variable_query_count_function()

def test_better_than_nothing(self):
    self.assertGoodView('first-view')
    self.assertGoodView('second-view')
    self.assertGoodView('something-else')

  
  
   What's this do? It gets the view at the named URL, ensures the status code is 200 and tests that the number of queries run is less than 50. Oh and it returns the response if you want it for other purposes. Not amazing, but better than having no test coverage.
  
  
   The future
  
  
   I've been using various versions of this module for years, but there is always room for improvement. Happy to take pull requests submissions for new methods that will be generally useful. Or maybe one day I'll be less lazy and see about merging these into Django core.
  
  
   Hope you like the module and find it useful. Happy Testing!

Improve your Django Tests using django-test-plus' helpful additions to the standard Django TestCase

Improved Django Tests

Four simple steps you can take to improve the performance of your Django applications

<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 
 Optimizing for performance often comes with a bunch of headaches and hard problems, but it doesn&rsquo;t always have to be that way.
 
 
 Below are four simple things you can do to quickly and easily improve the performance of your sites. They&rsquo;re so simple you should make them part of your standard setup.
 
 <h2>
 Persistent database connections
 </h2>
 
 New in Django 1.6 Django has built-in database connection persistence. Most everyone uses PostgreSQL for their production systems, and while connecting to PG isn&rsquo;t particularly slow, it is something we can optimize.
 
 
 Without persistent connections every request to your site also makes a connection to your database, authenticates the user, etc. This can easily take as much as 20&ndash;75ms if your database is on another host even on a fast network.
 
 
 To setup connection persistence you add the
 <code>
 CONN_MAX_AGE
 </code>
 parameter to your
 <code>
 DATABASES
 </code>
 setting to look like this:
 
 <div class="codehilite">
 <pre><code>DATABASES = {
 'default': {
 'ENGINE': 'django.db.backends.postgresql_psycopg2',
 'NAME': 'whoohoodb',
 'CONN_MAX_AGE': 600,
 }
}
</code></pre>
 </div>
 
 With this,we&rsquo;ve setup persistent connections to stay alive for 10 minutes at a time. This helps to cut down on any possible memory leaks or a flaky connection causing a long term problem. You can certainly use a longer value, but I wouldn&rsquo;t go much beyond an hour as there will be little benefit.
 
 
 You can read more details about
 <a href="https://docs.djangoproject.com/en/1.8/ref/databases/#persistent-connections">
 persistent database connections
 </a>
 in the Django documentation.
 
 <h2>
 Template Loading
 </h2>
 
 By default Django comes configured to use the two standard template loaders:
 
 <div class="codehilite">
 <pre><code>TEMPLATE_LOADERS = (
 'django.template.loaders.filesystem.Loader',
 'django.template.loaders.app_directories.Loader',
)
</code></pre>
 </div>
 
 These loaders search the file system and parse your templates on every request. Kinda surprising that it works as quickly as it does right?
 
 
 You can turn on cached loading, so Django only has to find and parse your templates one time, by simply changing your configuration to be:
 
 <div class="codehilite">
 <pre><code>TEMPLATE_LOADERS = (
 ('django.template.loaders.cached.Loader', (
 'django.template.loaders.filesystem.Loader',
 'django.template.loaders.app_directories.Loader',
 )),
)
</code></pre>
 </div>
 
 Now, don&rsquo;t be silly and turn on cached loading in your development environments as it will get annoying. You&rsquo;ll be forced to restart runserver (or whatever you&rsquo;re using locally) on each and every template change.
 
 <h2>
 Optimize Django Sessions
 </h2>
 
 If you poll Django users nearly 82% of them have no idea where their sessions are stored. Ok, I admit I made that number up, but in our experience it&rsquo;s pretty true.
 
 
 By default, Django stores your user sessions in your database and expects you to occasionally prune out old entries. Very few people do this.
 
 
 So on each request you&rsquo;re doing a SQL query to get the session data and another to grab the User object information. If you&rsquo;re the sort of person who&rsquo;s concerned about performance you likely already have memcached or Redis setup to cache some things. You can switch to storing your sessions in your cache and easily remove a SQL query from every single request to your site with:
 
 <div class="codehilite">
 <pre><code>SESSION_ENGINE = 'django.contrib.sessions.backends.cache'
</code></pre>
 </div>
 
 Now this assumes you can safely lose any data you store in your user sessions. If that&rsquo;s not the case, you can still get some benefit from using:
 
 <div class="codehilite">
 <pre><code>SESSION_ENGINE = 'django.contrib.sessions.backends.cached_db'
</code></pre>
 </div>
 
 This backend caches your data, but also writes any changes to your database for long term storage.
 
 <h3>
 select_related() and prefetch_related()
 </h3>
 
 The first two tips above were super easy, right? Unfortunately this last one, while still easy compared to totally changing up your data storage, does require a bit more than a single configuration setting change.
 
 
 <code>
 select_related()
 </code>
 and
 <code>
 prefetch_related()
 </code>
 aren&rsquo;t in everyone&rsquo;s bag of tricks but they are incredibly useful for cutting down on the number of SQL queries your ORM code is firing.
 
 
 The situation that usually happens is you have some
 <code>
 BlogPost
 </code>
 model with a ForeignKey to User. Then in a list view you use
 <code>
 queryset = BlogPost.objects.active
 </code>
 . Then in your templates you have something like:
 
 <div class="codehilite">
 <pre><code>&lt;ul&gt;
{% for post in object_list %}
 &lt;li&gt;{{ post.title }} - {{ post.user.email }}&lt;/li&gt;
{% endfor %}
&lt;/ul&gt;
</code></pre>
 </div>
 
 This ends up doing the expected SELECT against the blog_post table, but then is forced in the template loop to do another SELECT against auth_user
 
 for each and every post in the list
 
 .
 
 
 Instead, we can ask the Django ORM to go ahead and JOIN in the auth_user table to our results so that
 <code>
 object.user
 </code>
 is already an object for us. Taking us from
 <code>
 BlogPost.objects.active().count()
 </code>
 queries down to a single query.
 
 
 We can fix it by simply changing the view&rsquo;s queryset to be:
 
 <div class="codehilite">
 <pre><code>queryset = BlogPost.objects.select_related().active()
</code></pre>
 </div>
 
 While not a simple settings change, still pretty simple to accomplish.
 
 
 <code>
 prefetch_related
 </code>
 is the same mechanism, but for ManyToManyField relationships. Prefetch can&rsquo;t use a join here, so the
 
 join
 
 happens in Python at the ORM level. This can be a HUGE performance win for situations where you have a small number of rows in your ManyToManyField() vs a large number of rows in the model itself. However, in general, it&rsquo;s less useful and thus used less often that
 <code>
 select_related()
 </code>
 .
 
 
 So when do you use these you ask? The easiest thing to do is fire up
 <a href="https://django-debug-toolbar.readthedocs.org/en/1.3/">
 django-debug-toolbar
 </a>
 with your project and some representative data. By representative I mean you shouldn&rsquo;t have a single BlogPost and a single User, try more like 100+ of each.
 
 
 Then all you need to do is browse around your app locally with the debug toolbar open looking specifically at the query counts and times. If you see anything over 5&ndash;10 queries going on this is your clue to investigate.
 
 
 Most of the time with 100s of objects in your database you&rsquo;ll stumble across a view that is doing 200&ndash;500+ queries where one of these two options will be a huge win for you. Change your queryset, refresh, and make sure that both your query counts decreased but also your overall time. Occasionally you&rsquo;ll have a situation where it&rsquo;s actually faster to do the extra queries vs using joins it happens rarely, but still worth checking the overall query time to make sure you aren&rsquo;t making things worse.
 
 </div>
</div>

DATABASES = {
 'default': {
 'ENGINE': 'django.db.backends.postgresql_psycopg2',
 'NAME': 'whoohoodb',
 'CONN_MAX_AGE': 600,
 }
}

TEMPLATE_LOADERS = (
 'django.template.loaders.filesystem.Loader',
 'django.template.loaders.app_directories.Loader',
)

TEMPLATE_LOADERS = (
 ('django.template.loaders.cached.Loader', (
 'django.template.loaders.filesystem.Loader',
 'django.template.loaders.app_directories.Loader',
 )),
)

SESSION_ENGINE = 'django.contrib.sessions.backends.cache'

SESSION_ENGINE = 'django.contrib.sessions.backends.cached_db'

<ul>
{% for post in object_list %}
 <li>{{ post.title }} - {{ post.user.email }}</li>
{% endfor %}
</ul>

queryset = BlogPost.objects.select_related().active()

 
 
 While not a simple settings change, still pretty simple to accomplish.
 
 
 
 prefetch_related
 
 is the same mechanism, but for ManyToManyField relationships. Prefetch can’t use a join here, so the
 
 join
 
 happens in Python at the ORM level. This can be a HUGE performance win for situations where you have a small number of rows in your ManyToManyField() vs a large number of rows in the model itself. However, in general, it’s less useful and thus used less often that
 
 select_related()
 
 .
 
 
 So when do you use these you ask? The easiest thing to do is fire up
 
 django-debug-toolbar
 
 with your project and some representative data. By representative I mean you shouldn’t have a single BlogPost and a single User, try more like 100+ of each.
 
 
 Then all you need to do is browse around your app locally with the debug toolbar open looking specifically at the query counts and times. If you see anything over 5–10 queries going on this is your clue to investigate.
 
 
 Most of the time with 100s of objects in your database you’ll stumble across a view that is doing 200–500+ queries where one of these two options will be a huge win for you. Change your queryset, refresh, and make sure that both your query counts decreased but also your overall time. Occasionally you’ll have a situation where it’s actually faster to do the extra queries vs using joins it happens rarely, but still worth checking the overall query time to make sure you aren’t making things worse.

Django Performance: 4 Simple Things

Is Wagtail the best Django CMS? Review of Wagtail 1.0 features and our experiences using it.

from django.db import models
from wagtail.wagtailcore.models import Page
from wagtail.wagtailcore.fields import RichTextField
from wagtail.wagtailadmin.edit_handlers import FieldPanel

class BlogPage(Page):
    sub_title = models.CharField(max_length=500, blank=True)
    published = models.DateField()
    author = models.CharField(max_length=100)
    summary = models.RichTextField(blank=True)
    body = models.RichTextField()
    closing_content = models.RichTextField(blank=True)

    content_panels = [
        FieldPanel(‘title’),
        FieldPanel(‘sub_title’),
        FieldPanel(‘published’),
        FieldPanel(‘author’),
        FieldPanel(‘summary’),
        FieldPanel(‘body’),
        FieldPanel(‘closing_content’)
    ]

DEPARTMENT_CHOICES = (
    (‘admin’, ‘Administration’),
    (‘accounting’, ‘Accounting’),
    (‘marketing’, ‘Marketing’),
    (‘sales’, ‘Sales’),
    (‘engineer’, ‘Engineering’),
)

class StaffPage(Page):
    first_name = models.CharField(max_length=50)
    last_name = models.CharField(max_length=50)
    active = models.BooleanField(default=True)
    start_date = models.DateField()
    end_date = models.DateField(blank=True, null=True)
    department = models.CharField(max_length=50, choices=DEPARTMENT_CHOICES)
    email = models.EmailField()
    twitter = models.CharField(max_length=50)
    short_bio = models.RichTextField(blank=True)
    bio = models.RichTextField(blank=True)
    education = models.RichTextField(blank=True)
    work_history = models.RichTextField(blank=True)

    # Panel options left out for brevity

  
  
   As you can see the StaffPage model has quite a bit more fields, most of them optional, which allows the Staff member to update their information over time and not get strangled into putting ‘Bio coming soon’ into an otherwise required field.
  
  
   Pretty simple right? You’re probably thinking there is some catch, luckily you’re wrong. It’s really pretty much that simple. Easy things easy right?
  
  
   Harder things in Wagtail
  
  
   So what are the
   
    hard
   
   things in Wagtail? Well it’s mostly just getting familiar with the system in general. A few things that may trip you up are:
  
  
   
    You can’t have a field named
    
     url
    
    on your derived models as Wagtail uses that field name in the parent Page model. Unfortunately if you do add one, which I’ve done more times than I care to admit, you get a not very useful error “
    
     can’t set attribute
    
    ” and not much else to go on.
   
   
    On many listing type pages it’s fine to simply show all of the items, with pagination, in some sort of chronological order. Other times users want to be able to manually curate what shows up on given pages. Wagtail makes this relatively easy as you can define a ForeignKey relationship using a through type model to other pages and use a
    
     PageChoosePanel
    
    to give the user a nice interface for doing this. The user can also manually order them right in the admin, no additional work necessary.
   
   
    Limiting which pages can be created as children (aka beneath) a page can be handled by setting a list named
    
     parent_page_types
    
    on the child model. Then it can only be added below pages of those defined types. On complex sites with lots of different page types this helps keep the Page Choosing and Creation option choices to a manageable level for the users. And it also obviously helps to keep users from creating the wrong types of pages in the wrong parts of the site.
   
   
    Wagtail currently doesn’t have a great story around building navigation menus for you, but there are a dozen reusable Django apps to help handle that. Often a site’s menu remains relatively static and isn’t edited day-to-day anyway.
   
   
    Supporting multiple sites with the same CMS. This isn’t technically hard, but more conceptually difficult to grok. Wagtail supports having multiple sites, via it’s wagtailsites app. The way this works is you simply set the Root page for each hostname and it basically takes it from there. However, in most circumstances it’s probably easier and cleaner to just have two different instances of Wagtail and use different databases.
   
  
  
   Images and Documents
  
  
   Documents are files of any type you want to be able to upload into the system. This handles any sort of situations where a user needs to upload a PDF, Excel, or Word document and be able to link to it from any other bit of content.
  
  
   Images are exactly what you think, however you can define your own base model for this if you choose and attach additional fields for things like copyright, license, attribution, or even something like EXIF data if you wanted.
  
  
   With both Documents and Images having tagging support via
   
    django-taggit
   
   and a really nicely designed look and UX for them in the admin interface.
  
  
   And yes, before you ask, it has built in support for flexible thumbnails in your templates andthe ability for you to mainually define the main focal point in the image to avoid cropping things weirdly.
  
  
   
  
  
   Form Builder interface
  
  
   Wagtail also has a nice
   
    form builder
   
   built into it that can easily suffice for your typical contact form scenarios or more complicated collection needs.
  
  
   Much like Pages, you simply subclass from Wagtail and define what fields you want to collect. On your model you can also override the
   
    process_form_submission
   
   method to do more complex validation or in a more common case to email the interested parties that there is a new submission.
  
  
   One great feature of the form builder that is also built in, is the viewing and downloading interface. Viewing the data that has come in is great, but you just know your users are going to want to pull it out and use it for some other purpose. Wagtail smartly anticipates this and allows the user to download the submitted data, by date range, as a CSV file anytime they want.
  
  
   Snippets
  
  
   
    Wagtail Snippets
   
   are reusable bits of content that aren’t full web pages. Often these are used for things like sidebar content, advertisements, or calls to action.
  
  
   Unlike with Pages or Forms, you don’t subclass a model but instead define a model and simply register it as a snippet. You’re then free to give your users the option of attaching snippets of the types you want to other pages. Or if you just want to give them the ability to edit things like footer content for example, you can just manually include the snippet data and the Snippet admin UI is really just becomes their editing interface.
  
  
   Best Feature? Content Streams
  
  
   While being able to define your own Page types with their own fields goes a long way, it’s quite a stretch from truly free form content. New in version 1.0 is Wagtail’s killer feature the
   
    StreamField
   
   .
  
  
   Users want free form content while developers, designers, and even ops want nicely structured data. StreamFields satisfies both camps.
  
  
   When you define a StreamField on a Page you set what types of
   
    blocks
   
   are available to be added into that stream. A block can be something as simple as a CharField of text or as complicated as a Staff type record like above using
   
    structural block types
   
   .
  
  
   The end user can then add say few headers of various types, some rich text content blocks and have it all interspersed with a few images and code blocks. Each of these block types you define can then be styled differently CSS and/or have their markup structured entirely differently if needed.
  
  
   Prior to this feature being added to 1.0, I had to resort to complicated Page relationships that weren’t actually pages we intended to make visible on the site. We just subverted the Page Choosing features of Wagtail to give the users the flexibility they needed and keep it all in the same admin interface.
  
  
   Here is what the admin UI looks like for StreamFields. Here we've defined a field named Body that has header, content, and code block types. Each of these
   
    lines
   
   are different blocks. The top and bottom being headers. As you can see you can simply click the plus icons to add new blocks in between others or use the arrows on the right to move block around. They are currently a bit hard to see due to a CSS bug I anticipate being fixed quickly.
  
  
   
  
  
   Wagtail’s Future
  
  
   I think Wagtail has a
   
    VERY
   
   bright future in general and especially inside the Django community. However, like any 1.0 product there are definitely some things I would like to see in future versions. The two main things I hope to see are:
  
  
   
    A community collection of high quality and flexible common Page and Block types to make most sites more of a Lego exercise than a coding one.
   
   
    The ability to more easily customize and control the Publish/Moderate/Save as Draft options that appear at the bottom of the screen while editing content. On many smaller sites or those with a flat workflow it should be trivial to make ‘Publish’ or ‘Submit for Moderation’ be the default action presented to the user.

Wagtail 1.0 (beta) best Django CMS?

How to load files into Django models, not via a file upload, but from a script or Python shell.

<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 
 I don&rsquo;t know about you, but I run into a situation every so often where I need to programmatically push a file from the local file system or remotely via a URL into a Django model. Using a Python shell, a one off script, or a Django management command.
 
 
 This should be easy, but with a typical web app you do it infrequently. Mostly I find we do it when we&rsquo;re converting someone&rsquo;s site from something else to Django and need to associate an image with another Content item for example.
 
 
 If you start Googling around for answers to how to do this you get the default docs for
 <a href="https://docs.djangoproject.com/en/dev/topics/http/file-uploads/">
 File Uploads
 </a>
 ,
 <a href="https://docs.djangoproject.com/en/dev/ref/files/file/">
 File objects
 </a>
 , and
 <a href="https://docs.djangoproject.com/en/dev/topics/files/">
 Managing Files
 </a>
 which are all great. However, they don&rsquo;t
 
 exactly
 
 spell out how you&rsquo;re supposed to do this.
 
 
 My first inclination is to just assign the model field to a file, but this is missing the relative file path information Django needs.
 
 
 I took the time to write this post mostly for my own benefit so I don&rsquo;t have to figure this out yet again in the future, but hopefully you&rsquo;ll get some use out of it as well.
 
 <h2>
 Steps for loading Django ImageField with a local file
 </h2>
 
 Here is a quick example showing you the moving pieces. The steps you need to perform are:
 
 <ol>
 <li>
 Retrieve the file (if remote) and store it locally
 </li>
 <li>
 Open that file as a normal Python file object
 </li>
 <li>
 Covert that open file to a Django
 <a href="https://docs.djangoproject.com/en/dev/ref/files/file/">
 File
 </a>
 </li>
 <li>
 Attach it to your model field
 </li>
 </ol>
 <h2>
 Example Model
 </h2>
 
 Suppose we have a model like this:
 
 <div class="codehilite">
 <pre><code>from django.db import models

class Company(models.Model):
 name = models.CharField(max_length=100)
 logo = models.ImageField()
</code></pre>
 </div>
 
 Let&rsquo;s create an entry for RevSys, pulling the logo in with requests:
 
 <div class="codehilite">
 <pre><code>import requests
from django.core.files import File

from .models import Company

r = requests.get(&ldquo;http://media.revsys.com/img/revsys-logo.png&rdquo;)

with open(&ldquo;/tmp/revsys-logo.png&rdquo;, &ldquo;wb&rdquo;) as f:
 f.write(r.content)

reopen = open(&ldquo;/tmp/revsys-logo.png&rdquo;, &ldquo;rb&rdquo;)
django_file = File(reopen)

revsys = Company()
revsys.name = &ldquo;Revolution Systems&rdquo;
revsys.logo.save(&ldquo;revsys-logo.png&rdquo;, django_file, save=True)
</code></pre>
 </div>
 
 The last line here is the important bit.
 <code>
 save
 </code>
 is given three arguments here:
 
 <ol>
 <li>
 The relative path and filename for inside MEDIA_ROOT
 </li>
 <li>
 The open file using Django&rsquo;s File object
 </li>
 <li>
 Whether or not we want to save the revsys
 <code>
 Company
 </code>
 instance after the image is saved.
 </li>
 </ol>
 
 Now if you&rsquo;re doing this for a bunch of images, you&rsquo;ll want to parse your URLs or filenames to build the paths for you. Something like this would be a good starting point, note this is using Python 3:
 
 <div class="codehilite">
 <pre><code>import os
from urllib.parse import urlparse
# from urlparse import urlparse for Python 2.7

url = &ldquo;http://media.revsys.com/img/revsys-logo.png&rdquo;
filename = os.path.basename(urlparse(url).path)
# This returns &lsquo;revsys-logo.png&rsquo; from the URL
&hellip;

revsys.logo.save(filename, django_file, save=True)
</code></pre>
 </div>
 
 Hope this helps!
 
 </div>
</div>

from django.db import models

class Company(models.Model):
    name = models.CharField(max_length=100)
    logo = models.ImageField()

import requests
from django.core.files import File

from .models import Company

r = requests.get(“http://media.revsys.com/img/revsys-logo.png”)

with open(“/tmp/revsys-logo.png”, “wb”) as f:
    f.write(r.content)

reopen = open(“/tmp/revsys-logo.png”, “rb”)
django_file = File(reopen)

revsys = Company()
revsys.name = “Revolution Systems”
revsys.logo.save(“revsys-logo.png”, django_file, save=True)

import os
from urllib.parse import urlparse
# from urlparse import urlparse for Python 2.7

url = “http://media.revsys.com/img/revsys-logo.png”
filename = os.path.basename(urlparse(url).path)
# This returns ‘revsys-logo.png’ from the URL
…

revsys.logo.save(filename, django_file, save=True)

  
  
   Hope this helps!

Loading Django FileField and ImageFields from the file system

Our recommendation for the optimal Django project layout for your files, settings, and templates.

<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 <h2>
 What&rsquo;s the optimal layout for your Django applications, settings files, and various other associated directories?
 </h2>
 
 When Django 1.4 was released it included an
 <a href="https://docs.djangoproject.com/en/dev/releases/1.4/#updated-default-project-layout-and-manage-py">
 updated project layout
 </a>
 which went a long way to improving the default Django project&rsquo;s layout, but here are some tips for making it even better.
 
 
 This is a question we get asked all of the time so I wanted to take a bit of time and write down exactly how we feel about this subject so we can easily refer clients to this document. Note that this was written using Django version 1.7.1, but can be applied to any Django version after 1.4 easily.
 
 <h2>
 Why this layout is better
 </h2>
 
 The project layout we&rsquo;re recommending here has several advantages namely:
 
 <ul>
 <li>
 Allows you to pick up, repackage, and reuse individual Django applications for use in other projects. Often it isn&rsquo;t clear as you are building an app whether or not it is even a candidate for reuse. Building it this way from the start makes it much easier if that time comes.
 </li>
 <li>
 Encourages designing applications for reuse
 </li>
 <li>
 Environment specific settings. No more
 <code>
 if DEBUG==True
 </code>
 nonsense in a single monolithic settings file. This allows to easily see which settings are shared and what is overridden on a per environment basis.
 </li>
 <li>
 Environment specific PIP requirements
 </li>
 <li>
 Project level templates and static files that can, if necessary, override app level defaults.
 </li>
 <li>
 Small more specific test files which are easier to read and understand.
 </li>
 </ul>
 
 Assuming you have two apps
 
 blog
 
 and
 
 users
 
 and 2 environments dev and prod your project layout should be structured like this:
 
 <div class="codehilite">
 <pre><code>myproject/
 manage.py
 myproject/
 __init__.py
 urls.py
 wsgi.py
 settings/
 __init__.py
 base.py
 dev.py
 prod.py
 blog/
 __init__.py
 models.py
 managers.py
 views.py
 urls.py
 templates/
 blog/
 base.html
 list.html
 detail.html
 static/
 &hellip;
 tests/
 __init__.py
 test_models.py
 test_managers.py
 test_views.py
 users/
 __init__.py
 models.py
 views.py
 urls.py
 templates/
 users/
 base.html
 list.html
 detail.html
 static/
 &hellip;
 tests/
 __init__.py
 test_models.py
 test_views.py
 static/
 css/
 &hellip;
 js/
 &hellip;
 templates/
 base.html
 index.html
 requirements/
 base.txt
 dev.txt
 test.txt
 prod.txt
</code></pre>
 </div>
 
 The rest of this article explains how to move a project to this layout and why this layout is better.
 
 <h2>
 Current Default Layout
 </h2>
 
 We&rsquo;re going to call our example project
 
 foo
 
 , yes I realize it&rsquo;s a very creative name. We&rsquo;re assuming here that we&rsquo;re going to be launching
 
 foo.com
 
 but while we like to have our project names reflect the ultimate domain(s) the project will live on this isn&rsquo;t by any means required.
 
 
 If you kick off your project using
 <code>
 django-admin.py startproject foo
 </code>
 you get a directory structure like this:
 
 <div class="codehilite">
 <pre><code> foo/
 manage.py
 foo/
 __init__.py
 settings.py
 urls.py
 wsgi.py
</code></pre>
 </div>
 
 This layout is a great starting place, we have a top level directory
 
 foo
 
 which contains our manage.py the
 
 project
 
 directory
 
 foo/foo/
 
 inside it. This is the directory you would check into your source control system such as git.
 
 
 You should think of this
 
 foo/foo/
 
 subdirectory as being
 
 the project
 
 where everything else is either a Django application or ancillary files related to the project.
 
 <h2>
 Fixing Settings
 </h2>
 
 We&rsquo;re on a mission to fix your bad settings files here. We show this layout to new clients and I&rsquo;m constantly surprised how few people know this is even possible to do. I blame the fact that while everyone knows that settings are just Python code, they don&rsquo;t think about them
 
 as
 
 Python code.
 
 
 So let&rsquo;s fix up our settings. For our foo project we&rsquo;re going to have 4 environments: dev, stage, jenkins, and production. So let&rsquo;s give each it&rsquo;s own file. The process to do this is:
 
 <ol>
 <li>
 In
 
 foo/foo/
 
 make a settings directory and create an empty
 <code>
 __init__.py
 </code>
 file inside it.
 </li>
 <li>
 Move
 
 foo/foo/settings.py
 
 into
 
 foo/foo/settings/base.py
 
 </li>
 <li>
 Create the individual
 
 dev.py
 
 ,
 
 stage.py
 
 ,
 
 jenkins.py
 
 , and
 
 prod.py
 
 files in
 
 foo/foo/settings/
 
 . Each of these 4 environment specific files should simply contain the following:
 <div class="codehilite">
 <pre><code>from base import *
</code></pre>
 </div>
 </li>
 </ol>
 
 So why is this important? Well for local development you want
 <code>
 DEBUG=True
 </code>
 , but it&rsquo;s pretty easy to accidentally push out production code with it on, so just open up
 
 foo/foo/settings/prod.py
 
 and after the initial import from base just add
 <code>
 DEBUG=False
 </code>
 . Now if your production site is safe from that silly mistake.
 
 
 What else can you customize? Well it should be pretty obvious you&rsquo;ll likely have staging, jenkins, and production all pointing at different databases, likely even on different hosts. So adjust those settings in each environment file.
 
 <h2>
 Using these settings
 </h2>
 
 Using these settings is easy, no matter which method you typically use. To use the OS&rsquo;s environment you just do:
 
 <div class="codehilite">
 <pre><code>export DJANGO_SETTINGS_MODULE=&ldquo;foo.settings.jenkins&rdquo;
</code></pre>
 </div>
 
 And boom, you&rsquo;re now using the jenkins configuration.
 
 
 Or maybe you prefer to pass them in as a commandline option like this:
 
 <div class="codehilite">
 <pre><code>./manage.py migrate &mdash;settings=foo.settings.production
</code></pre>
 </div>
 
 Same if you&rsquo;re using gunicorn:
 
 <div class="codehilite">
 <pre><code>gunicorn -w 4 -b 127.0.0.1:8001 &mdash;settings=foo.settings.dev
</code></pre>
 </div>
 <h2>
 What else should be customized about settings?
 </h2>
 
 Another useful tip with Django settings is to change several of the default settings
 
 collections
 
 from being tuples to being lists. For example
 <code>
 INSTALLED_APPS
 </code>
 , by changing it from:
 
 <div class="codehilite">
 <pre><code>INSTALLED_APPS = (
 &hellip;
)
</code></pre>
 </div>
 
 to:
 
 <div class="codehilite">
 <pre><code>INSTALLED_APPS = [
 &hellip;
]
</code></pre>
 </div>
 
 In
 
 foo/settings/base.py
 
 we can now more easily add and remove apps based on each environment specific settings file. For example, maybe you only want django-debug-toolbar installed in dev, but not your other environments.
 
 
 This trick is also often useful for the
 <code>
 TEMPLATE_DIRS
 </code>
 and
 <code>
 MIDDLEWARE_CLASSES
 </code>
 settings.
 
 
 Another useful trick we often use is to break up your apps into two lists, one your prerequisites and another for your actual project applications. So like this:
 
 <div class="codehilite">
 <pre><code>PREREQ_APPS = [
 &lsquo;django.contrib.auth&rsquo;,
 &lsquo;django.contrib.contenttypes&rsquo;,
 &hellip;
 &lsquo;debug_toolbar&rsquo;,
 &lsquo;imagekit&rsquo;,
 &lsquo;haystack&rsquo;,
]

PROJECT_APPS = [
 &lsquo;homepage&rsquo;,
 &lsquo;users&rsquo;,
 &lsquo;blog&rsquo;,
]

INSTALLED_APPS = PREREQ_APPS + PROJECT_APPS
</code></pre>
 </div>
 
 Why is this useful? For one it helps better distinguish between Django core apps, third party apps, and your own internal project specific applications. However,
 <code>
 PROJECT_APPS
 </code>
 often comes in handy as a list of your specific apps for things like testing and code coverage. You have a list of
 
 your
 
 apps, so you can easily and automagically make sure their tests are run and coverage is recorded just for them, not including any third party apps, without having to maintain the list in two separate places.
 
 <h2>
 Fixing requirements
 </h2>
 
 Most projects have a single
 <code>
 requirements.txt
 </code>
 file that is installed like this:
 
 <div class="codehilite">
 <pre><code>pip install -r requirements.txt
</code></pre>
 </div>
 
 This is sufficient for small simple projects, but a little known feature of requirements files is that you can use the
 <code>
 -r
 </code>
 flag to include other files. So we can have a base.txt of all the common requirements and then if we need to be able to run tests have a specific
 
 requirements/test.txt
 
 that looks like this:
 
 <div class="codehilite">
 <pre><code>-r base.txt
pytest==2.5.2
coverage==3.7.1
</code></pre>
 </div>
 
 I&rsquo;ll admit this is not a
 
 HUGE
 
 benefit, but it does help separate out what is a requirement in which environment. And for the truly performance conscience it reduces your pip install time in production a touch by not installing a bunch of things that won&rsquo;t actually be used in production.
 
 <h2>
 Test Files
 </h2>
 
 Why did we separate out the tests files so much? One main reason, if you&rsquo;re writing enough tests a single tests.py file per application will end up being one huge honking file. This is bad for readability, but also just for the simple fact you have to spend time scrolling around a lot in your editor.
 
 
 You&rsquo;ll also end up with less merge conflicts when working with other developers which is a nice side benefit. Small files are your friends.
 
 <h2>
 URLs
 </h2>
 
 For small projects it&rsquo;s tempting to put all of your url definitions in
 
 foo/urls.py
 
 to keep them all in one place. However, if your goal is clarity and reusability you want to define your urls in each app and include them into your main project. So instead of:
 
 <div class="codehilite">
 <pre><code>urlpatterns = patterns(&lsquo;&rsquo;,
 url(r&rsquo;^$&rsquo;, HomePageView.as_view(), name=&lsquo;home&rsquo;),
 url(r&rsquo;^blog/$&rsquo;, BlogList.as_view(), name=&lsquo;blog_list&rsquo;),
 url(r&rsquo;^blog/(?P&lt;pk&gt;\d+)/$&rsquo;, BlogDetail.as_view(), name=&lsquo;blog_detail&rsquo;),
 &hellip;
 url(r&rsquo;^user/list/$&rsquo;, UserList.as_view(), name=&lsquo;user_list&rsquo;),
 url(r&rsquo;^user/(?P&lt;username&gt;\w+)/$&rsquo;, UserDetail.as_view(), name=&lsquo;user_detail&rsquo;),
)
</code></pre>
 </div>
 
 you should do this:
 
 <div class="codehilite">
 <pre><code>urlpatterns = patterns(&lsquo;&rsquo;,
 url(r&rsquo;^$&rsquo;, HomePageView.as_view(), name=&lsquo;home&rsquo;),
 url(r&rsquo;^blog/&lsquo;, include(&lsquo;blog.urls&rsquo;)),
 url(r&rsquo;^user/&lsquo;, include(&lsquo;user.urls&rsquo;)),
)
</code></pre>
 </div>
 <h2>
 Templates and static media
 </h2>
 
 Having per app
 
 templates/
 
 and
 
 static/
 
 directories gives us the ability to reuse an application basically
 
 as is
 
 in another project.
 
 
 We get the default templates the app provides and any associated static media like special Javascript for that one cool feature all in one package.
 
 
 However, it
 
 also
 
 gives us the ability to override those templates on a per project basis in the main
 
 foo/templates/
 
 directory. By adding a
 <code>
 templates/blog/detail.html
 </code>
 template we override, or mask, the default
 <code>
 blog/templates/blog/detail.html
 </code>
 template.
 
 <h2>
 Reusing a Django application
 </h2>
 
 So assuming you&rsquo;ve been using this layout for awhile, one day you&rsquo;ll realize that your new project needs a blog and the one from your
 
 foo
 
 project would be perfect for it. So you copy and paste the files in&hellip;
 
 pssst WRONG!
 
 . Now you have two copies of the application out there. Bug fixes or new features in one have to manually be moved between the projects, and that assumes you even remember to do that.
 
 
 Instead, make a new repo for your blog and put the
 
 foo/blog/
 
 directory in it. And adjust both your existing foo project and your new project to pip install it.
 
 
 They can still both track different versions of the app, if necessary, or keep up to date and get all of your bug fixes and new features as they develop. You still can override the templates and static media as you need to on a per project basis, so there really isn&rsquo;t any real issues doing this.
 
 <h2>
 Additional Resources
 </h2>
 
 Our friends Danny and Audrey over at
 <a href="http://www.cartwheelweb.com">
 CartWheel Web
 </a>
 reminded us about
 <a href="https://github.com/audreyr/cookiecutter">
 Cookie Cutter
 </a>
 and specifically Danny's
 <a href="https://github.com/pydanny/cookiecutter-django">
 cookiecutter-django
 </a>
 as useful tools for making your initial project creations easier and repeatable.
 
 
 Also, if you're looking for all around great Django tips and best practices, you can't go wrong with their book
 <a href="http://www.amazon.com/gp/product/098146730X/ref=as_li_qf_sp_asin_tl?ie=UTF8&amp;camp=1789&amp;creative=9325&amp;creativeASIN=098146730X&amp;linkCode=as2&amp;tag=revosystblog-20">
 Two Scoops of Django: Best Practices For Django 1.6
 </a>
 <img alt="" src="http://ir-na.amazon-adsystem.com/e/ir?t=revosystblog-20&amp;l=as2&amp;o=1&amp;a=098146730X"/>
 which we recommend to all of our clients.
 
 <h2>
 Feedback
 </h2>
 
 We hope you find this improved project layout useful. If you find any bugs, have a suggestion, or just want to chat feel free to reach out to us. Thanks for reading!
 
 </div>
</div>

myproject/
 manage.py
 myproject/
 __init__.py
 urls.py
 wsgi.py
 settings/
 __init__.py
 base.py
 dev.py
 prod.py
 blog/
 __init__.py
 models.py
 managers.py
 views.py
 urls.py
 templates/
 blog/
 base.html
 list.html
 detail.html
 static/
 …
 tests/
 __init__.py
 test_models.py
 test_managers.py
 test_views.py
 users/
 __init__.py
 models.py
 views.py
 urls.py
 templates/
 users/
 base.html
 list.html
 detail.html
 static/
 …
 tests/
 __init__.py
 test_models.py
 test_views.py
 static/
 css/
 …
 js/
 …
 templates/
 base.html
 index.html
 requirements/
 base.txt
 dev.txt
 test.txt
 prod.txt

 foo/
 manage.py
 foo/
 __init__.py
 settings.py
 urls.py
 wsgi.py

from base import *

export DJANGO_SETTINGS_MODULE=“foo.settings.jenkins”

./manage.py migrate —settings=foo.settings.production

gunicorn -w 4 -b 127.0.0.1:8001 —settings=foo.settings.dev

INSTALLED_APPS = (
 …
)

INSTALLED_APPS = [
 …
]

PREREQ_APPS = [
 ‘django.contrib.auth’,
 ‘django.contrib.contenttypes’,
 …
 ‘debug_toolbar’,
 ‘imagekit’,
 ‘haystack’,
]

PROJECT_APPS = [
 ‘homepage’,
 ‘users’,
 ‘blog’,
]

INSTALLED_APPS = PREREQ_APPS + PROJECT_APPS

pip install -r requirements.txt

-r base.txt
pytest==2.5.2
coverage==3.7.1

urlpatterns = patterns(‘’,
 url(r’^$’, HomePageView.as_view(), name=‘home’),
 url(r’^blog/$’, BlogList.as_view(), name=‘blog_list’),
 url(r’^blog/(?P<pk>\d+)/$’, BlogDetail.as_view(), name=‘blog_detail’),
 …
 url(r’^user/list/$’, UserList.as_view(), name=‘user_list’),
 url(r’^user/(?P<username>\w+)/$’, UserDetail.as_view(), name=‘user_detail’),
)

urlpatterns = patterns(‘’,
 url(r’^$’, HomePageView.as_view(), name=‘home’),
 url(r’^blog/‘, include(‘blog.urls’)),
 url(r’^user/‘, include(‘user.urls’)),
)

 
 
 Templates and static media
 
 
 Having per app
 
 templates/
 
 and
 
 static/
 
 directories gives us the ability to reuse an application basically
 
 as is
 
 in another project.
 
 
 We get the default templates the app provides and any associated static media like special Javascript for that one cool feature all in one package.
 
 
 However, it
 
 also
 
 gives us the ability to override those templates on a per project basis in the main
 
 foo/templates/
 
 directory. By adding a
 
 templates/blog/detail.html
 
 template we override, or mask, the default
 
 blog/templates/blog/detail.html
 
 template.
 
 
 Reusing a Django application
 
 
 So assuming you’ve been using this layout for awhile, one day you’ll realize that your new project needs a blog and the one from your
 
 foo
 
 project would be perfect for it. So you copy and paste the files in…
 
 pssst WRONG!
 
 . Now you have two copies of the application out there. Bug fixes or new features in one have to manually be moved between the projects, and that assumes you even remember to do that.
 
 
 Instead, make a new repo for your blog and put the
 
 foo/blog/
 
 directory in it. And adjust both your existing foo project and your new project to pip install it.
 
 
 They can still both track different versions of the app, if necessary, or keep up to date and get all of your bug fixes and new features as they develop. You still can override the templates and static media as you need to on a per project basis, so there really isn’t any real issues doing this.
 
 
 Additional Resources
 
 
 Our friends Danny and Audrey over at
 
 CartWheel Web
 
 reminded us about
 
 Cookie Cutter
 
 and specifically Danny's
 
 cookiecutter-django
 
 as useful tools for making your initial project creations easier and repeatable.
 
 
 Also, if you're looking for all around great Django tips and best practices, you can't go wrong with their book
 
 Two Scoops of Django: Best Practices For Django 1.6
 
 
 which we recommend to all of our clients.
 
 
 Feedback
 
 
 We hope you find this improved project layout useful. If you find any bugs, have a suggestion, or just want to chat feel free to reach out to us. Thanks for reading!

Recommended Django Project Layout

Simple way to use virtaualenvwrapper&#x27;s postactivate and deactivate hooks to execute arbitrary Python code for any of your Python shells while working on that project.

<div class="block-markdown">
 <div class="prose prose-lg max-w-none">
 <h2>
 Do you find yourself re-doing the same Python shell stuff?
 </h2>
 
 I know I do. I'm constantly doing it and replaying some of my IPython history isn't cutting it for me anymore. IPython has
 <a href="http://ipython.org/ipython-doc/dev/config/intro.html#profiles">
 profiles
 </a>
 but they're not terribly easy to get working with Django and your typical
 <code>
 manage.py shell
 </code>
 command.
 
 <h2>
 Site wide is easy, but what about per project?
 </h2>
 
 You can load things and execute code for
 
 ALL
 
 your Python shells system wide pretty easily, but what about when you need different things loaded depending on the project?
 
 
 Turns out it's pretty easy to do with an environment variable and
 <a href="http://virtualenvwrapper.readthedocs.org/en/latest/">
 virtualenvwrapper
 </a>
 . If you aren't already using virtualenvwrapper you should really check it out. I use it even outside of Python projects just to have an easy mechanism for different "shell environments" to have certain things executed when I start and finish working on a project.
 
 <h2>
 The situation I was in...
 </h2>
 
 While the issue that lead me to figure this out was a Django project, nothing about this technique is Django specific. I have a pretty common workflow when doing green field Django development where I setup Fabric tasks to rebuild my local dev database and repopulate it with certain database objects I need to make the site functional enough for testing. For example, I typically setup a User object with username 'frank' (I know big surprise there) that is a superuser. So if I launch a shell 9 out of 10 times I probably need my
 
 frank
 
 User object for something.
 
 
 This is complicated however by the fact that on some projects I work on the User model is customized to not have a username so in some projects I need to run:
 
 <div class="codehilite">
 <pre><code>frank = User.objects.get(username='frank')
</code></pre>
 </div>
 
 And in others I need:
 
 <div class="codehilite">
 <pre><code>frank = User.objects.get(email='frank@revsys.com')
</code></pre>
 </div>
 
 What I realized was Python honors and executes the code in any file you specify in the
 <code>
 PYTHONSTARTUP
 </code>
 environment variable in your shell. So if you adjust your virtualenv's
 <a href="http://virtualenvwrapper.readthedocs.org/en/latest/plugins.html#plugins-post-activate">
 postactivate and postdeactivate
 </a>
 hooks to set the path to code you want run on a per project basis. For example:
 
 <h3>
 In ~/.virtualenvs/projectX/bin/postactivate
 </h3>
 <div class="codehilite">
 <pre><code> #!/bin/bash
 export PYTHONSTARTUP=/Users/frank/work/src/projectX/startup.py
 export DJANGO_SETTINGS_MODULE="projectX.settings.dev"
 cd /Users/frank/work/src/projectX
</code></pre>
 </div>
 <h3>
 In ~/.virtualenvs/projectX/bin/postdeactivate
 </h3>
 <div class="codehilite">
 <pre><code> #!/bin/bash
 unset $DJANGO_SETTINGS_MODULE
 unset $PYTHONSTARTUP
</code></pre>
 </div>
 <h2>
 So what can you do with this?
 </h2>
 
 Well you can execute any arbitrary Python code so you could could do a lot, even go so far as to hit your time tracking software's API to log the time, but realize anything you run will slow down the startup time of each Python shell you launch so keep it to what you really need. On this particular project I kept it to a few small things:
 
 <div class="codehilite">
 <pre><code> # Import datetime and some of my commonly used Models 
 import datetime
 from django.contrib.auth import get_user_model
 from app1.models import ThatOneModel
 from app2.models import TheOtherCommonModel

 try:
 User = get_user_model()
 frank = User.objects.get(username='frank')
 except User.DoesNotExist:
 # Frank not being in the DB shouldn't raise an error
 pass
</code></pre>
 </div>
 
 Hope this helps you stay in flow more and delay your carpal tunnel a few days at least!
 
 </div>
</div>

frank = User.objects.get(username='frank')

frank = User.objects.get(email='frank@revsys.com')

  #!/bin/bash
  export PYTHONSTARTUP=/Users/frank/work/src/projectX/startup.py
  export DJANGO_SETTINGS_MODULE="projectX.settings.dev"
  cd /Users/frank/work/src/projectX

  #!/bin/bash
  unset $DJANGO_SETTINGS_MODULE
  unset $PYTHONSTARTUP

  # Import datetime and some of my commonly used Models 
  import datetime
  from django.contrib.auth import get_user_model
  from app1.models import ThatOneModel
  from app2.models import TheOtherCommonModel

  try:
      User = get_user_model()
      frank = User.objects.get(username='frank')
  except User.DoesNotExist:
      # Frank not being in the DB shouldn't raise an error
      pass

  
  
   Hope this helps you stay in flow more and delay your carpal tunnel a few days at least!

Simple way to use virtaualenvwrapper's postactivate and deactivate hooks to execute arbitrary Python code for any of your Python shells while working on that project.

Hello, Python

Python Jobs 💼