Updated March 24, 2023
Introduction to Python Threadpool
With the passage of time, the data involved in a program has increased exponentially, and this has led to the adaptation of new techniques, which reduces the execution time of a program. Out of this need for faster program execution, the concept of Concurrency came into being. In this topic, we are going to learn about Python Threadpool.
The literal meaning of the word Concurrency is a simultaneous occurrence. Likewise, the concept of Concurrency is about parallel computation, and thus it decreases the execution time of a program. In Python, there are mainly three simultaneously occurring entities, namely thread, task, and processes.
Before discussing the main topic, let us first have a brief understanding of Threads and Threading.
Threads: A Thread is a component of a Process that can run parallely. There can be multiple threads inside a parent process. All the threads share the program to be executed and the data required for it within the parent process.
Threading: Threading is a library in Python that helps to achieve parallel programming with the various threads residing inside the parent process.
Working of Threading
Threading’s fundamental unit is a thread, multiple of which can reside inside a parent process, and each one accomplishes a separate task. The thread object first needs to be created and initialized by passing the function name and the arguments.
Then to start a particular thread, the start() function is required, and the join() function indicates that the execution of that thread is complete. Each thread needs its separate session for execution. Since request.Session() is not thread-safe; one thread can access a piece of code or memory at one time, and it is achieved by threading.Lock.
Thread Pool in Python
In Python, a Thread Pool is a group of idle threads pre-instantiated and are ever ready to be given the task.
We can either instantiate new threads for each or use Python Thread Pool for new threads. But when the number of tasks is way more than Python Thread Pool is preferred over the former method.
A thread pool can manage parallel execution of a large number of threads as follows: –
- A thread can be reused if a thread in a thread pool completes its execution.
- A new thread is created to replace a thread that is terminated.
How to use Python Threadpool?
concurrent.futures is a module present in the Python standard library. It contains a concrete subclass known as ThreadPoolExecuter, which uses multi-threading, and we get a pool of thread for submitting the tasks. The pool thus created assigns tasks to the available threads and scheduled them to run.
Let us see the syntax of Thread Pool Executor to better understand its working: –
Syntax: –
from concurrent.futures import ThreadPoolExecutor
from time import sleep
def func_name(arguements):
// function definition
def main():
executor = ThreadPoolExecutor(num_of_threads)
future = executor.submit(function_name, (arguement))
print(future.done())
sleep(2)
print(future.done())
print(future.result())
if __name__ == '__main__':
main()Examples of Python Threadpool
Here are the Examples of Python Threadpool mention below
Example #1
Code:
from concurrent.futures import ThreadPoolExecutor
from time import sleep
def count_number_of_words(sentence):
number_of_words = len(sentence.split())
sleep(1)
print("Number of words in the sentence :",sentence," : {}".format(number_of_words),end="\n")
def count_number_of_characters(sentence):
number_of_characters = len(sentence)
sleep(1)
print("Number of characters in the sentence :",sentence," : {}".format(number_of_characters),end="\n")
if __name__ == '__main__':
sentence = "Python Multiprocessing is an important library for achieving parallel programming."
executor = ThreadPoolExecutor(4)
thread1 = executor.submit(count_number_of_words, (sentence))
thread2 = executor.submit(count_number_of_characters, (sentence))
print("Thread 1 executed ? :",thread1.done())
print("Thread 2 executed ? :",thread2.done())
sleep(2)
print("Thread 1 executed ? :",thread1.done())
print("Thread 2 executed ? :",thread2.done())Output:
Code Explanation: In the above example, a Thread Pool Executor has been created with 4 threads. Then two tasks which are signified by the functions count_number_of_words and count_number_of_characters, respectively, will wait for 1 second each before executing the functions and displaying the result. The tasks do not complete in the first one-second interval, so the call to the done() function returns a False value. After the task is executed and the respective print statements are displayed, then again, when the done() function is called, it returns a true value.
Example #2
Code:
import concurrent.futures
import numpy as np
from time import sleep
numbers = [10,23,54,7,89,100]
def get_max_number(numbers):
greatest_num = np.max(numbers)
sleep(2)
print("Greatest number is :{}".format(greatest_num))
with concurrent.futures.ThreadPoolExecutor(max_workers = 4) as executor:
thread1 = executor.submit(get_max_number, (numbers))
print("Thread 1 executed ? :",thread1.done())
sleep(2)
print("Thread 1 executed ? :",thread1.done())Output:
Code Explanation: This example shows the Context Manager’s use to instantiate the ThreadPoolExecuter, with the help of which we have created 4 threads. Then the task, which is signified by the function get_max_number(arguments), will wait for 2 seconds before executing the function and displaying the result. The tasks do not complete in the first two-second interval, so the call to the done() function returns a False value. After the task is executed and the respective print statements are displayed, then again, when the done() function is called, it returns a true value.
Example #3
Code:
from concurrent.futures import ThreadPoolExecutor
from concurrent.futures import as_completed
import numpy as np
from time import sleep
def log(n):
log_value = np.log(n)
sleep(1)
return log_value
if __name__ == '__main__':
values = [1,10,100,1000]
with ThreadPoolExecutor(max_workers = 3) as executor:
thread1 = executor.map(log, values)
for result in thread1:
print(np.round(result,2))Output:
Code Explanation: This example shows the use of the Executor.map function has been displayed. We know in Python, a map function is used to apply a certain function to every element within iterables. Here we have mapped all the elements of the iterator values to the function named log(argument) and have submitted these as independent jobs to the ThreadPoolExecutor.
Conclusion
In the following article, we have discussed the fundamentals of Python Threadpool and how it works internally. We saw at the syntax of Python Thread Pool along with 3 examples to better understand the concept. So next time you stumble upon a program that requires parallel computation, does remember to use threads and use ThreadPoolExecutor to better appreciate the use of them.
Recommended Articles
This is a guide to Python Threadpool. Here we discuss the basic concept, how to use a Python Threadpool? along with the respective examples. You may also have a look at the following articles to learn more –
