- Python 3.7
- Django 2.1
ManyToManyFields confuse a lot of people. The way you relate objects to each other using a many-to-many relationship is just different enough from dealing with
ForeignKeys and just uncommon enough in day-to-day Django development that it's easy to forget all the little tricks for dealing with them.
When should you use a
ManyToManyField instead of a regular
ForeignKey? To remember that, let's think about pizza. A pizza can have many toppings (a Hawaiian pizza usually has Canadian bacon and pineapple), and a topping can go on many pizzas (Canadian bacon also appears on meat lovers' pizzas). Since a pizza can have more than one topping, and a topping can go on more than one pizza, this is a great place to use a
So let's dive in: assume the following models in a
from django.db import models class Pizza(models.Model): name = models.CharField(max_length=30) toppings = models.ManyToManyField('Topping') def __str__(self): return self.name class Topping(models.Model): name = models.CharField(max_length=30) def __str__(self): return self.name
Both objects must exist in the database
You have to save a
Topping in the database before you can add it to a
Pizza, and vice versa. This is because a
ManyToManyField creates an invisible "through" model that relates the source model (in this case
Pizza, which contains the
ManyToManyField) to the target model (
Topping). In order to create the connection between a pizza and a topping, they both have to be added to this invisible "through" table. From the Django docs:
" [T]here is … an implicit through model class you can use to directly access the table created to hold the association. It has three fields to link the models. If the source and target models differ, the following fields are generated:
id: the primary key of the relation.
<containing_model>_id: the id of the model that declares the ManyToManyField.
<other_model>_id: the id of the model that the ManyToManyField points to."
The invisible "through" model that Django uses to make many-to-many relationships work requires the primary keys for the source model and the target model. A primary key doesn't exist until a model instance is saved, so that's why both instances have to exist before they can be related. (You can't add spinach to your pizza if you haven't bought spinach yet, and you can't add spinach to your pizza if you haven't even started rolling out the crust yet either.)
See what happens when you try to add a topping to a pizza before you've added that topping to the database:
>> from pizzas.models import Pizza, Topping >> hawaiian_pizza = Pizza.objects.create(name='Hawaiian') >> pineapple = Topping(name='pineapple') >> hawaiian_pizza.toppings.add(pineapple) Traceback (most recent call last): ... ValueError: Cannot add "<Topping: pineapple>": instance is on database "default", value is on database "None" >>
ValueError is raised because the
pineapple hasn't yet been saved, so its value on the database doesn't exist yet. But when I save
pineapple, I can add it to my pizza.
>> pineapple.save() >> hawaiian_pizza.toppings.add(pineapple) >> hawaiian_pizza.toppings.all() <QuerySet [<Topping: pineapple>]>
The reverse doesn't work either: I can't create a topping in the database, and then add it to a pizza that hasn't been saved.
>> pepperoni = Topping.objects.create(name='pepperoni') >> pepperoni_pizza = Pizza(name='Pepperoni') >> pepperoni_pizza.toppings.add(pepperoni) Traceback (most recent call last): ... ValueError: "<Pizza: Pepperoni>" needs to have a value for field "id" before this many-to-many relationship can be used.
This error is more explicit (it states that an id is required) but it's essentially the same error. It's just coming from the other side of the relationship.
To retrieve the stuff in a
ManyToManyField, you have to use
Since the field
toppings is already on the
Pizza model, getting all the toppings on a specific pizza is pretty straightforward.
>> hawaiian_pizza.toppings.all() <QuerySet [<Topping: pineapple>, <Topping: Canadian bacon>]>
But if I try to see what pizzas use Canadian bacon, I get an
>> canadian_bacon.pizzas.all() Traceback (most recent call last): ... AttributeError: 'Topping' object has no attribute 'pizzas'
That's because Django automatically refers to the target
ManyToManyField objects as "sets." The pizzas that use specific toppings are in their own "set":
>> canadian_bacon.pizza_set.all() <QuerySet [<Pizza: Hawaiian>]>
… unless you add the
related_name option to the field
related_name option to a
ManyToManyField will let you choose a more intuitive name to use when you want to retrieve the stuff in that field.
class Pizza(models.Model): ... toppings = models.ManyToManyField('Topping', related_name='pizzas')
related_name should usually be the lowercase, plural form of your model name. This is confusing for some people because shouldn't the
toppings just be… toppings?
related_name isn't referring to how you want to retrieve the stuff in this field; it specifies the term you want to use instead of
*_set when you're on the target object (which in this case is a topping) and want to see which source objects point to that target (what pizzas use a specific topping).
related_name, we would retrieve all the pizzas that use a specific topping with
>> canadian_bacon.pizza_set.all() <QuerySet [<Pizza: Hawaiian>]>
related_name of "pizzas" to the
toppings attribute lets us retrieve all the toppings for a pizza like this:
>> canadian_bacon.pizzas.all() <QuerySet [<Pizza: Hawaiian>]>
You can add things from both sides of the relationship.
Earlier, we created a
Pizza object, and then a
Topping object, and then ran
hawaiian_pizza.toppings.add(pineapple) to associate the pineapple topping with the Hawaiian pizza.
But we could, instead, add a pizza to a topping.
>> cheese_pizza = Pizza.objects.create(name='Cheese') >> mozzarella = Topping.objects.create(name='mozzarella') >> mozzarella.pizzas.add(cheese_pizza) >> mozzarella.pizzas.all() <QuerySet [<Pizza: Cheese>]>
You can query the items in the many-to-many set from both sides
Say we want to find all the pizzas that have toppings that start with the letter "p." We can write that query like this:
>> Pizza.objects.filter(toppings__name__startswith='p') <QuerySet [<Pizza: Pepperoni Pizza>, <Pizza: Hawaiian Pizza>]>
A Hawaiian pizza contains pineapple, and a pepperoni pizza contains pepperonis. Pineapple and pepperoni both start with the letter "p," so the both of those pizzas are returned.
We can do the same from the
Topping model, to find all the toppings used on pizzas that contain "Hawaiian" in their name:
>> Topping.objects.filter(pizzas__name__contains='Hawaiian') <QuerySet [<Topping: pineapple>, <Topping: Canadian bacon>]>
You might need a custom "through" model
Remember when I mentioned the invisible "through" model that Django creates to manage your many-to-many relationships? You might want to keep track of more data about those relationships, and to do that you would use a custom "through" model.
The example used in the Django docs is of a Group, Person, and Membership relationship. A group can have many people as members, and a person can be part of many groups, so the
Group model has a
ManyToManyField that points to
Person. Then, a
Membership model contains
ForeignKeys to both
Group, and can store extra information about a person's membership in a specific group, like the date they joined, who invited them, etc.
But we're not here to talk about people. We are all about pizza.
Using our existing models, we can create all kinds of pizzas with a wide range of toppings. But we can't make a pizza like "Super Pepperoni" that contains double the usual amount of pepperonis. We can't add pepperoni to a pizza more than once:
>> pepperoni_pizza.toppings.all() <QuerySet [<Topping: pepperoni>]> >> pepperoni_pizza.toppings.add(pepperoni) >> pepperoni_pizza.toppings.all() <QuerySet [<Topping: pepperoni>]>
Django just ignores us if we try. A "through" model would let us specify a quantity for each topping, enabling us to add "pepperoni" once, but specify that we wanted twice the amount for the Super Pepperoni pizza.
Note: If you're going to use a "through" model, you have to start with that in mind… or be willing to either drop your database or do some very advanced database finagling. If you try to add a "through" model later, you will see an error like this one when you run
ValueError: Cannot alter field pizzas.Pizza.toppings into pizzas.Pizza.toppings - they are not compatible types (you cannot alter to or from M2M fields, or add or remove through= on M2M fields)
If you're following along, now is the time to drop your database, delete your migration files, and add this new model:
class ToppingAmount(models.Model): REGULAR = 1 DOUBLE = 2 TRIPLE = 3 AMOUNT_CHOICES = ( (REGULAR, 'Regular'), (DOUBLE, 'Double'), (TRIPLE, 'Triple'), ) pizza = models.ForeignKey('Pizza', related_name='topping_amounts', on_delete=models.SET_NULL, null=True) topping = models.ForeignKey('Topping', related_name='topping_amounts', on_delete=models.SET_NULL, null=True, blank=True) amount = models.IntegerField(choices=AMOUNT_CHOICES, default=REGULAR)
Now, add the
through option to the
toppings field on the
class Pizza(models.Model): ... toppings = models.ManyToManyField('Topping', through='ToppingAmount', related_name='pizzas')
If we try to add a pizza, a topping, and then associate them the way we used to, we will get an error:
>> super_pep = Pizza.objects.create(name='Super Pepperoni') >> pepperoni = Topping.objects.create(name='pepperoni') >> super_pep.toppings.add(pepperoni) Traceback (most recent call last): ... AttributeError: Cannot use add() on a ManyToManyField which specifies an intermediary model. Use pizzas.ToppingAmount's Manager instead.
Using a custom "through" model forces us to use that model to associate the pizza and toppings.
>>> super_pep_amount = ToppingAmount.objects.create(pizza=super_pep, topping=pepperoni, amount=ToppingAmount.DOUBLE)
But the benefit is that we can now add some extra information about that relationship, like the fact that the amount of pepperonis on a Super Pepperoni pizza should be double the regular amount.
We can still access the toppings from the pizza:
>> super_pep.toppings.all() <QuerySet [<Topping: pepperoni>]>
We'll only see pepperoni once, since the amount is on the "through" model. And we can access the pizzas that use a specific topping:
>> pepperoni.pizzas.all() <QuerySet [<Pizza: Super Pepperoni>]>
But now, we can use our "through" model to get all the toppings and their amount for a specific pizza from the
for top_amt in ToppingAmount.objects.filter(pizza=super_pep): print(top_amt.topping.name, top_amt.get_amount_display()) pepperoni Double
You can also see the topping amounts from the
Pizza objects themselves.
for top_amt in super_pep.topping_amounts.all(): print(top_amt.topping.name, top_amt.get_amount_display()) pepperoni Double
And you can also access the amount for a specific topping from the
topping on the
for topping in super_pep.toppings.all(): for top_amt in topping.topping_amount.all(): print(topping.name, top_amt.amount, top_amt.get_amount_display()) pepperoni 2 Double
You could extend this
ToppingAmount through model to hold information about the left and right halves of the pizza, or notes about topping preparation ("put peppers under cheese"). You can also add methods to the through model or the source/target models to more easily access some of the topping amount information.
A through model is also useful for relationships between players and teams; the through model could contain information about the players' positions, jersey numbers, and dates they joined the team. A through model joining movie theatres and films could contain the number of screens the film is showing on and the start and end run dates. Students' relationships to their Degree Programs could track information like GPA, whether the program is the student's major or minor, whether it's a double major, and start/end semesters the student was in a program.
- The many-to-many relationships section of the Django docs contains a lot of great examples of using
- The explanation of
ManyToManyFieldin the model field reference.
- Jacob Kaplan-Moss also has a great set of examples of using a many-to-many relationship with a custom "through" model.