Python Flask Multithreading

Introduction

We will go over a Python AWS Boto3 Load Balancer Guide.

Did you know that there’s way to use threads within Flask?

We will break down this in the following sections:

• Is flask multithreaded
• Can you add threads in flask
• Does flask support multiple clients
• How to run Flask tasks in the background

I have used this successfully in various projects and it works very well and has saved me a ton of time getting going with messing around with background tasks in Flask.

We will go point by point with some code snipets on how to achieve this and also elaborate to your main questions regarding Flask.

Is Python Flask Multithreaded

The Python Flask framework is multi-threaded by default. This change took place in Version 1.0 where they introduced threads to handle multiple new requests. Using this the Flask application works like this under the hood:

• Network request comes in from a client online
• Flask accepts the connection and registers a request object
• Before it does any processing flask registers the connection internally to it’s queue to process
• Once this done Flask creates a lightweight thread to handle the request and keeps track of it in a thread pool
• The thread is consumed from the pool and processed using the standard Python interpreter
• Scheduling and prioritization of thread consumption is entirely dependent on how Python handles it

As you can see above the process is pretty standardized and it’s typically what most Python programs do. The important part to note is that you are still at the mercy of the Python scheduler and you can’t do much about that. Furthermore you are relying on the GIL (Global Interpreter Lock more info below), to activate or deactivate specific action on each of your threads. So CPU time is given typically based on code that needs to be executed or if I/O operations are going to take place. I found that the CPU time is allocated fairly well and generally things are pretty much a smooth sail without any particular problems.

Can You Serve Multiple Clients Using Flask

The answer is yes. As we discussed earlier Python Flask now offers threading out of the box so you do not need to do anything to activate it. The catch here is to understand that it still has the limitations of any other Python application that uses threading. Since Python is not by default a multi-process language things tend to context switch so everything gets executed within one CPU.

To better visual this I have created an image showing how the event bridge in flask handles multiple clients. Keep in mind this is network based on events so it’s slightly different at that level on how to works behind the scenes with the Linux threading scheduler.

This means that when you invoke the Python interpreter it is certain that no other processes will fork from it to run on a separate CPU unless you explicitly use something like multi-processing. Having said that this handicaps a bit the capabilities of Python Flask in terms of performance when it relies on scaling a CPU intensive job as everything else has to wait for it’s completion (asynchronously). Despite this typically you want to put your long lasting jobs in a separate thread with your own control or send it as a separate background task using Celery.

In the next section we will discuss this specific thing how to run a multi-threaded operation inside Flask.

Can You Use Multithreading Inside Flask

The short answer is yes you can use multi-threading inside a Flask application. However there is a catch. When you are using thread locals those are not supported very well at the moment which means if you need to do any synchronization or information sharing between your threads you will have a lot of difficulty achieving it.

If you reference directly from the Flask design document as of now for version 2.2.x here’s the snipet they say:

Flask uses thread local objects (context local objects in fact, they support greenlet contexts as well) for request, session and an extra object you can put your own things on. Why is that and isn’t that a bad idea? Yes it is usually not such a bright idea to use thread locals. They cause troubles for servers that are not based on the concept of threads and make large applications harder to maintain. However Flask is just not designed for large applications or asynchronous servers. Flask wants to make it quick and easy to write a traditional web application.

From the above you can easily determine that Flask was not designed with threading in mind however do not let this deter you below we will demonstrate some code snipets on how you can achieve this. Having said that keep in mind that you are doing something against the design principles of an application, which may lead to problems. In my testing the method that I will discuss about below works fine and have not had any issues, with the exception that I go into great lengths not to rely on locals or generally use mutexes on my data stores. In fact the way I use threading is entirely to run a task in the background without having to wait for it, more like a fire-and-forget approach.

The problem gets worst if you try to make something of a thread pool and need synchronization. Think about the scenario that pretty much most of Flask threads are run and forget, this makes it difficult to manage things across them as there’s no shared resource for the information. Complicating the logic and adding another layer into this, will likely negate all the advantages you previously had from using threads to begin with. This is a word of caution from my experience to avoid it and just stick to the Flask model based on handling network events.

How To Background Task In Flask

So now that we have gone over on why the design of Flask is not ideal for threads we will discuss how to background a task in Flask using Python threading. The code snipet below will be a good first step on how to do this:

The first step we want to do is define a threaded function that we will be executing as a background Python task. To do this we will be inheriting the Thread class from the threading Python library and will define our new run function that has the code we want to execute.

In the example below we keep this empty and just print out a message to ensure our task executed in the background successfully. In a real case scenario you want to populate this with your own code before the print out function so it can execute whatever you need.

import threading
 
class NewThreadedTask(threading.Thread):
     def __init__(self):
         super(NewThreadedTask, self).__init__()
 
     def run(self):
         # run some code here
         print('Threaded task has been completed')

Now that we have defined our threaded Python function all we have to do is start defining our main Flask application and invoking our threaded call. To do this we will be implementing a code snipet as shown below.

@app.route('/api/background-task', methods=['GET'])
def background_task():
    new_thread = NewThreadedTask()
    new_thread.start()
    # optionally wait for task to complete
    new_thread.join()
    return {'status': 'ok'}, 200

Lets go over line by line and see what we did in the code above:

• First we define a new route as /api/background-task, that’s the prefix path for the main flask URL to append too
• The methods we will be accepting for now is simply a GET request to demonstrate our threading example
• Our implementation function basically works by first initializing an instance of the new threaded task class we defined earlier (it inherits from the parent multi threading library of Python).
• Then we simply start it by invoking the start function
• We optionally wait for the function to complete using join, note here if you want to fire and forget and don’t care of what happens to it you don’t need to do this. However you need to keep note that when you try to terminate the Flask application and if your thread is still running you need to wait for it using a join or you will risk having unterminated threads. Generally it’s a good idea that you put all invoked thread objects in a pool and monitor if they are still running and haven’t expired/crashed. This will prevent a memory leak and give you more control of what’s happening in your threads. It will further allow you to monitor the threads by polling their status and other things.
• Finally once the function completes we simply return a 200 with a custom JSON status code indicating that everything completed ok in the execution of the task.

There’s a few things that we need to be careful about here and those come with every Python multi threaded application:

• All threads need to complete prior you terminate the Flask application as mentioned above
• Any thread synchronization needs to take place prior termination as well so if your threads are doing important database operations or something similar you need to make sure that gets cleaned up gracefully.
• Threads are two edge swords if you are relying on a common data store and you are using mutexes be extremely careful to release the mutex after a while or you may end up in a deadlock situation and your whole application will lock up. Even worst if your Flask application has spawned a thread and it’s waiting for it to join and complete then that Flask thread that’s serving the web request will also lock up.

To avoid these problems make sure you implement and approach this using Python threading best practises the most important of which I talked about earlier.

As of late 2023 Flask has made even more improvements but I still do not recommend to people to use it as such unless really needed. Obviously based on your use case you may find yourself doing that if you need to run some tasks asynchronously inside your API request serving just make sure if there’s synchronization you do not end up with a deadlock which is a very common problem.

Conclusion

We were able to successfully go over a Python Flask Multithreading, hopefully I answered any questions you may have had and helped you get started on your quest on using threads in flask.

If you found this useful and you think it may have helped you please drop me a cheer below I would appreciate it.

If you have any questions, comments please post them below or send me a note on my twitter. I check periodically and try to answer them in the priority they come in. Also if you have any corrections please do let me know and I’ll update the article with new updates or mistakes I did.

Do you prefer to use multithreading or multiprocessing?

I personally still think both have their place and complement each other whenever I can get away I like to stick to thread greenlets as they are very lightweight and perform pretty well if the blocker is I/O. If you have CPU blockers then you may want to consider using processes as they use a different thread to execute your code. In the case of flask most things would be solved with threading.

If you would like to find more articles related to AWS services:

• Which Python Library Should You Learn First
• Debug Multiprocessing In Python
• How to setup and use Django Rest Framework (Python 3)
• How To Create REST API Using API Gateway
• How To Stress Test REST API Using Python
• Is Flask Beginner Friendly

Here is a list of some links that may provide reference for you:

• Flask Design
• Python multi threading
• Python GIL