Introduction
Today I’m going to talk about how to do unit testing in Django Rest Framework (Python 3). I have to say here this is a continuation of a previous article I wrote on how to setup a Django Rest API from scratch using Django rest framework. If you haven’t read it yet I highly recommend you do before you go through this next step on how to test your app. You can find more information on it here. If you have already read it then great I hope you have your first app and running successfully today I’m going to teach you how to do unit testing on your APIs automatically without having to go and test them manually across releases. This is something you can add in your Jenkins/Gitlab or testing framework prior releasing a new version.
This automated unit testing has saved me a lot of hours of work not only by testing the integrity of my code base but also finding issues/bugs I introduced in my libraries without realizing that may have affected some API end points. Like I said above pairing it as part of a deployment workflow is where it works wonders. If you don’t have one don’t worry we will cover how to do this manually before you do a git push and save you from potential embarrassment or late nights debugging issues after they make it into a production environment.
A little about me, I have been working in the Software industry for over 23 years now and I have been a software architect, manager, developer and engineer. I am a machine learning and crypto enthusiast with emphasis in security. I have experience in various industries such as entertainment, broadcasting, healthcare, security, education, retail and finance. I have used Django and later the Django Rest Framework since it’s inception and I have seen it mature over time in a great way. I have to say it’s my personal favorite these days.
In this article I’m going to break down everything in sections of order on what you need to do in your code base to get the unit testing working. You can also find everything I will talk about here in a git repository I made specifically for this project here. So without saying too much more lets just dive into it.
How to setup the Environment in Django Rest Framework
How to Install pip packages for Django Rest Framework
We will need one extra package here that will help us a bit with the visualization of when running the unit testing. The package is ‘redgreenunittest’. Once installed this will require some further configuration which we will discuss in the settings category later in this document. Please don’t forget to activate your virtual environment if you haven’t done so prior installing the pip package.
Once activated please go ahead and install it as shown below:
main ●✚ (env) alex@DYNAMH ~/code/unbiased-coder/python-django-rest-framework-guide > pip install redgreenunittest
Collecting redgreenunittest
Using cached redgreenunittest-0.1.1-py3-none-any.whl
Requirement already satisfied: Pygments in ./env/lib/python3.8/site-packages (from redgreenunittest) (2.10.0)
Installing collected packages: redgreenunittest
Successfully installed redgreenunittest-0.1.1
WARNING: You are using pip version 21.2.3; however, version 21.2.4 is available.
You should consider upgrading via the '/home/alex/code/unbiased-coder/python-django-rest-framework-guide/env/bin/python.exe -m pip install --upgrade pip' command.How to Configure Settings in Django Rest Framework
How to Setup Test Runner in Django Rest Framework
Now that we have the package dependencies installed we can go ahead and start editing our settings file to turn on a few things that weren’t there previously.
First we need to configure the unit testing python runner tool, you can do this by adding this snipet in your settings file:
TEST_RUNNER = "redgreenunittest.django.runner.RedGreenDiscoverRunner"This sets the unit testing test runner to the redgreen package we installed previously that will make the output of our testing more readable.
How to Setup Logging in Django Rest Framework
Another thing we will be introducing to our project is some form of debug logging. Since we need a little more control over the output messages in order to debug problems we will be introducing python’s logger. Django has a nice way of doing this and even allows you to configure some settings such as the default logging level and handlers among others. To keep things simple in this article we will simply log everything in our console and set the default level to WARNINGS and above. Do note the print outs we are introducing in our code are by default in the DEBUG level so you will not be able to see them when running our unit testing unless you change the WARNING level to DEBUG since DEBUGGING is lower in the hierarchy of the Python logger.
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'console': {
'class': 'logging.StreamHandler',
},
},
'root': {
'handlers': ['console'],
'level': 'WARNING',
},
}
How to setup Django Rest Framework Renderer
One last configuration change we need to do in our settings is to add a default django rest framework section that defines a renderer. The reason we want to do is to ensure the Django Rest Framework returns our results into raw JSON. As you already know Django in general has different ways of returning rendered content such as html, templating etc. By turning this on our testing framework can work with raw JSON which will allow us to display the results easily. You can add more renderers if you wish to support them in the configuration below but to make it easier I just include what is necessary.
REST_FRAMEWORK = {
'TEST_REQUEST_DEFAULT_FORMAT': 'json',
'TEST_REQUEST_RENDERER_CLASSES': [
'rest_framework.renderers.JSONRenderer',
]
}Now that that we have our environment setup properly configured we can move on to start doing some fun stuff. If for any reason stuff is not working properly please visit the git repo I have uploaded all the files there that you can use as reference during this setup.
How to unit test listing objects in Django Rest Framework
As discussed above I will walk you through testing all the aspects of your REST API. This means listing, editing, adding and deleting items from your collection. I believe it’s crucial you test all aspects that your endpoint supports. Now I understand that not all endpoints will offer these 4 options so feel free to adjust accordingly and remove if needed but I’m going to include an example for all 4 in case you do have them. I will start with the easiest one which is listing objects.
One important thing to understand about the Django Unit testing framework is that everything is running inside an isolated context. What this means is that any objects you create, delete, edit will expire as soon as testing is completed. Also each testing case has it’s own context. So lets assume you are adding a new entry in the database in one of your tests then your other test that does the listing will not see it! This means that if you are trying to list items you need to add some inside your test before you do so. There are ways to keep consistency but we are not going to discuss this here for three reasons:
- In production environments and real testing you do not want to pollute your database with testing junk
- You are not truly testing as there could be interdependencies in your data so it’s basically better to create the unit test from scratch, otherwise it would be a functional test and not a unit test.
- Tests run much faster when you start with an empty context.
In order to write a test we will be modifying the default tests.py file that’s inside your apps. Similar to the models and serializers there’s also a unit testing specific in the application context which is part of the project scope. The code below demonstrates how this can be done, I wrote a helper function to populate the database with some data. Also note we will be re-using the example of the previous article I wrote about how to setup a Django Rest Framework so this is a continuation to it. If you would like to just focus on the testing keep reading otherwise you can refer to it for more information here.
from rest_framework.reverse import reverse
from rest_framework import status
from rest_framework.test import APITestCase
from app.models import Person
import datetime
import logging
logger = logging.getLogger(__name__)
class PersonViewSetTests(APITestCase):
def add_test_person(self):
"""
Adds a test person into the database
"""
logger.debug('Adding a new person into database')
p = Person(first_name='Unbiased', last_name='Coder', dob=datetime.date(2021, 9, 1))
p.save()
logger.debug('Successfully added test person into the database')
def test_list_persons(self):
"""
Test to list all the persons in the list
"""
logger.debug('Starting test list persons')
self.add_test_person()
url = 'https://unbiased-coder.com:8000%s'%reverse('person-list')
logger.debug('Sending TEST data to url: %s'%url)
response = self.client.get(url, format='json')
json = response.json()
logger.debug('Testing status code response: %s, code: %d'%(json, response.status_code))
self.assertEqual(response.status_code, status.HTTP_200_OK)
logger.debug('Testing result count')
self.assertEqual(len(json), 1)We are performing two unit tests in the above:
- Making sure the API end point returns the proper http code (200) in this case
- Making sure it’s returning 1 item that we previously add in the same code
In here we are leveraging the equivalents of the vanilla Django code when it comes to the client, the url retrieval etc. The same code would have worked in vanilla Django the only difference is that you would have to import the helper functions from the Django library paths rather than the ones from the rest framework.
How to unit test adding an object in Django Rest Framework
As shown above we wrote a unit test to retrieve records from the database and ensure the count is correct. We could have added more test cases such as verifying the details we added were correct etc. Similar to above now I will show how you can write a unit test to add new objects into the database via your REST framework. It must be noted here we’re testing all of those as you would normally do from the web client. There’s options that you can add to your client api that also allows you to do csrf and authentication with it. To keep things simple here I omitted those things but you can find more information about it here.
def test_create_person(self):
"""
Tests creating a new person object
"""
logger.debug('Starting test create person')
url = 'https://unbiased-coder.com:8000%s'%reverse('person-list')
data = {
'first_name' : 'Unbiased',
'last_name' : 'Coder',
'dob' : '2021-09-01'
}
logger.debug('Sending TEST data to url: %s, data: %s'%(url, data))
response = self.client.post(url, data, format='json')
logger.debug('Testing status code response: %s, code: %d'%(response.json(), response.status_code))
self.assertEqual(response.status_code, status.HTTP_201_CREATED)
logger.debug('Testing person count to make sure object was successfully added')
self.assertEqual(Person.objects.count(), 1)
logger.debug('Testing new person object details')
p = Person.objects.get()
self.assertEqual(p.first_name, 'Unbiased')
self.assertEqual(p.last_name, 'Coder')
self.assertEqual(p.dob, datetime.date(2021, 9, 1))
logger.debug('Test person create completed successfully')
The tests we are doing in the above code are the following:
- Making sure the http code is that of creating a new entry (HTTP 201)
- Making sure the count is 1 as we are only adding 1 record.
- Then we verify the details of the person are correct something we omitted to do in the previous example.
How to unit test deleting an object in Django Rest Framework
Without going into too many details here’s the code that does the same for deleting a record:
def test_delete_persons(self):
"""
Test to see if deleting works
"""
logger.debug('Starting test delete persons')
self.add_test_person()
url = 'https://unbiased-coder.com:8000%s1/'%reverse('person-list')
logger.debug('Sending TEST data to url: %s'%url)
response = self.client.delete(url, format='json')
logger.debug('Testing to see if status code is correct')
self.assertEqual(response.status_code, status.HTTP_204_NO_CONTENT)
Same as the listing test we are adding a sample person object and then proceed into deleting it. Since there’s no content in this response there’s nothing to verify other than the HTTP code to see if it worked and didn’t return any failures.
How to unit test editing an object in Django Rest Framework
Finally we will be editing a person object. The workflow of this is very similar to that of adding a new person but in this case we will be slightly be adjusting the person’s first and last name and then verifying it by seeing what response we got from the API.
def test_put_persons(self):
"""
Test to see if put works
"""
logger.debug('Starting test put persons')
self.add_test_person()
url = 'https://unbiased-coder.com:8000%s1/'%reverse('person-list')
logger.debug('Sending TEST data to url: %s'%url)
data = {
'first_name' : 'Unbiased11',
'last_name' : 'Coder11',
'dob' : '2021-09-01'
}
response = self.client.put(url, data, format='json')
json = response.json()
logger.debug('Testing to see if status code is correct')
self.assertEqual(response.status_code, status.HTTP_200_OK)
logger.debug('Testing modified person object details')
p = Person.objects.get()
self.assertEqual(p.first_name, 'Unbiased11')
self.assertEqual(p.last_name, 'Coder11')
logger.debug('Test person put completed successfully')
The tests we are doing above are the following:
- Making sure the response http code is correct (200)
- Verifying the name of the person was adjusted correctly in this case the first name from: Unbiased -> Unbiased11 and last name from: Coder to Coder11.
This last test concludes all the unit testing we need to be doing to fully test our REST API. As briefly mentioned above you can add support for authentication and csrf validation.
How to run Unit Testing in Django Rest Framework
Now that everything is in place the time for the most fun part has come. We will try to execute our code and see what output we will get from the unit testing. One thing to note here is that we will be leveraging the internal python test command to launch it. If we assume we will be using the API we implemented in the previous article, then everything should be fine and we should not get any errors. In order to get a little more verbose output we will be running the command with the -v 2 option. This will show us test by test the status and what happened rather than just a summary. If you find this information too overwhelming you can ommit passing the -v 2 argument to your command line.
So lets just go ahead and run our first test (this should be successful as we had previously tested it manually).
main ●✚ (env) alex@DYNAMH ~/code/unbiased-coder/python-django-rest-framework-guide > python manage.py test -v 2
Creating test database for alias 'default' ('file:memorydb_default?mode=memory&cache=shared')...
Operations to perform:
Synchronize unmigrated apps: messages, rest_framework, staticfiles
Apply all migrations: admin, app, auth, contenttypes, sessions
Synchronizing apps without migrations:
Creating tables...
Running deferred SQL...
Running migrations:
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying admin.0001_initial... OK
Applying admin.0002_logentry_remove_auto_add... OK
Applying admin.0003_logentry_add_action_flag_choices... OK
Applying app.0001_initial... OK
Applying contenttypes.0002_remove_content_type_name... OK
Applying auth.0002_alter_permission_name_max_length... OK
Applying auth.0003_alter_user_email_max_length... OK
Applying auth.0004_alter_user_username_opts... OK
Applying auth.0005_alter_user_last_login_null... OK
Applying auth.0006_require_contenttypes_0002... OK
Applying auth.0007_alter_validators_add_error_messages... OK
Applying auth.0008_alter_user_username_max_length... OK
Applying auth.0009_alter_user_last_name_max_length... OK
Applying auth.0010_alter_group_name_max_length... OK
Applying auth.0011_update_proxy_permissions... OK
Applying auth.0012_alter_user_first_name_max_length... OK
Applying sessions.0001_initial... OK
System check identified some issues:
System check identified 1 issue (0 silenced).
test_create_person (app.tests.PersonViewSetTests)
Tests creating a new person object ... ok
test_delete_persons (app.tests.PersonViewSetTests)
Test to see if deleting works ... ok
test_list_persons (app.tests.PersonViewSetTests)
Test to list all the persons in the list ... ok
test_put_persons (app.tests.PersonViewSetTests)
Test to see if put works ... ok
----------------------------------------------------------------------
Ran 4 tests in 0.023s
OK
As you can see above it executed 4 test tasks and they all completed successfully. But what happens if there’s a problem in our code? Lets find out!
We will be introducing a programming error in our view on purpose to see how the unit test is showing our failure. In order to do this we will be editing the serializers file and we will be removing one of the important attributes that is included in all of our requests, more specifically the date of birth (dob) of the Person object. The adjust code would look like this now:
from django.contrib.auth.models import User, Group
from app.models import Person
from rest_framework import serializers
class PersonSerializer(serializers.HyperlinkedModelSerializer):
class Meta:
model = Person
fields = ['first_name', 'last_name']
One thing to note here this error file is not in the git repository for obvious reasons so you will need to make the adjustment manually if you want to test it. Now that we introduced a problem lets try to re-run our testing framework and see what happens:
======================================================================
ERROR: test_create_person (app.tests.PersonViewSetTests)
Tests creating a new person object
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alex/code/unbiased-coder/python-django-rest-framework-guide/env/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/home/alex/code/unbiased-coder/python-django-rest-framework-guide/env/lib/python3.8/site-packages/django/db/backends/sqlite3/base.py", line 423, in execute
return Database.Cursor.execute(self, query, params)
sqlite3.IntegrityError: NOT NULL constraint failed: app_person.dob
----------------------------------------------------------------------
Ran 4 tests in 0.130s
FAILED (errors=1)
To avoid flooding you I abbreviated the output above to keep things simple. The part that’s important here is that we are seeing the first test which is trying to create a person is failing because the dob is missing from the serializer. As correctly identified by our testing framework we can now safely say our unit testing is working. If this was part of a testing workflow framework it would have prevented a deployment and saved you from an error.
Conclusion
I personally find unit testing very useful and think it’s an integral part of software development. If your team does not have a lot of money to hire a team of QA analysts then this might be your saving grace. Furthermore I think it actually complements the QA analysts work as it finds things that are sometimes missed from the basic UI testing that is done most of the time. In terms of time it’s a huge saver as you don’t have to manually audit your code to find such tedious problems. Now imagine having a framework with 300 API end points you will be basically lost without this. Today we discussed how to do unit testing in Django Rest Framework (Python 3) some features may overlap with vanilla Django do not hesitate to apply them there too!
If you found this article useful and you think it may have helped you please drop me a cheer below I would appreciate it.
If you have any questions, comments below I check periodically and try to answer them in the priority they come in.
Also if you have any corrections please do let me know and I’ll update the article with new updates or mistakes I did.
Do you write unit testing for your APIs?
If you enjoyed this you can check some similar articles I wrote here: