Updated April 3, 2023
Introduction to Python Concurrency
In today’s world, data is the new oil. The amount of data generated every second has increased exponentially than what used to be a decade back. So, the efficiency of a program is determined by how well it can manipulate and work on such huge data. Such a huge amount of data comes with a problem; it increases the program’s time of execution, thus affecting its efficiency. Thus, out of the need for faster execution of a program on huge sets of data, the concept of Concurrency in Python came into being. The literal meaning of the word Concurrency is a simultaneous occurrence. Likewise, the concept of Concurrency is about parallel computation, and thus it increases the execution time of a program.
What is Python Concurrency?
In Python, there are mainly three simultaneously occurring things, namely thread, task, and processes. From a high level, all three accomplish the same thing: reducing the time of execution of a program, but all of them are different from each other at a minute level. Though all the three forms of concurrency (i.e. processes, tasks, threads) aim to achieve concurrent execution, only multiprocessing achieves concurrent execution in the true sense. It executes multiple processes parallelly on multiple processors. Threading and asyncio both run on a single processor. Yet, they are considered to be concurrent processes since they speed up the execution process.
Types of Concurrency
1. Threading: Threading is also known as pre-emptive multitasking as the OS knows about each and every thread and can interrupt at any moment to start executing on another thread.
2. Asyncio: It is also known as cooperative multitasking as the tasks cooperate and decide when to give up control.
3. Multiprocessing: It achieves concurrent execution in the true sense as the processes run at the same time on different processors.
How Does Concurrency Works?
Below we learn how does concurrency works:
1. Threading
Threading’s fundamental unit is a thread, multiple of which can reside inside a parent process, and each one accomplishes a separate task. The thread object first needs to be created and initialized by passing the function name and the arguments. Each thread needs its separate session for execution since request.Session() is not thread-safe; one thread can access a piece of code or memory at one time, and it is achieved by threading.Lock.
Syntax:
import threading
def func_name(arguement):
// function definition
if__name__ == "__main__" :
# Thread creation
t1 = threading.Thread(target=func_name,args=( arguement,))
# Staring Thread t1
t1.start()
# Waiting until Thread t1 is completely executed.
t1.join()
2. Asyncio
The basic concept of asyncio is that a single Python object known as the event loop controls how and when each task gets run. The event loop is aware of each task and its state. The ready state indicates that the task is ready to run, and the waiting stage indicates that the task is waiting for some external task to complete. In asyncio tasks, never give up control and do not get interrupted in the middle of execution, so object sharing is safer in it than threading, and it is also thread-safe.
Syntax:
import asyncio
async def func_name(arguements..):
// function definition
async def main ():
# Creating n number of tasks
task1 = loop.create_task(func_name(arguement1))
task2 = loop.create_task(func_name(arguement2))
.
.
.
taskn = loop.create_task(func_name(arguementn))
await asyncio.wait([task1, task2,.., taskn])
if__name__ =='__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()
3. Multiprocessing
Multiprocessing achieves concurrency in its true sense as it executes code across different processes on different CPU cores. It creates a new instance of the Python interpreter to run on each CPU. Different processes reside in a different memory location, so object sharing among them is not so easy.
Syntax:
import multiprocessing
def func_name(parameters):
// function definition
if __name__ == "__main__":
# Creating process p1
p1 = multiprocessing.Process(target=func_name,args=(arguement1,))
# Creating process p2
p2 = multiprocessing.Process(target=func_name,args=(arguement2,))
# Starting process p1
p1.start()
# Starting process p2
p2.start()
# Waiting for process p1 to be completely executed
p1.join()
# Waiting for process p2 to be completely executed
p2.join()
Examples to Implement Concurrency in Python
Below are the examples of Concurrency in Python:
Example #1 – Threading
Code:
import threading
from datetime import datetime
def count_number_of_words(sentence):
number_of_words = len (sentence.split())
print("Number of words in the sentence :" ,sentence," : {}".format(number_of_words))
def count_number_of_characters(sentence):
number_of_characters = len (sentence)
print("Number of characters in the sentence :" ,sentence, " : {}".format(number_of_characters))
if__name__=="__main__":
sentence = "Python Multiprocessing is an important library for achieving parallel programming."
start = datetime.now().microsecond
t1 = threading.Thread(target=count_number_of_words,args=(sentence,))
t2 = threading.Thread(target=count_number_of_characters,args=(sentence,))
t1.start()
t2.start()
t1.join()
t2.join()
end = datetime.now().microsecond
print("Time taken to execute process : {}".format((end - start)/1000)," ms")
Output:
Example #2 – Asyncio
Code:
import asyncio
#from datetime import datetime
async def count_number_of_words(sentence):
print("Counting number of words in sentence : {}".format(sentence))
number_of_words = len(sentence.split())
located = []
for i in range(3):
located.append (i)
print("Done with counting number of words in sentence : {}".format(number_of_words))
return located
async def main ():
task1 = loop1.create_task(count_number_of_words("Asyncio is a way of achieving Concurrency in Python."))
task2 = loop1.create_task(count_number_of_words("It is not concurrency in true sense."))
task3 = loop1.create_task(count_number_of_words("It uses only one processor at a time."))
await asyncio.wait([task1, task2, task3])
if __name__ == ' __main__ ' :
loop1 = asyncio.get_event_loop()
loop1.run_until_complete(main())
loop1.close()
Output:
Example #3 – Multiprocessing
Code:
import multiprocessing
from datetime import datetime
def count_number_of_words(sentence):
number_of_words = len(sentence.split())
print("Number of words in the sentence :",sentence," : {}".format(number_of_words))
def count_number_of_characters(sentence):
number_of_characters = len(sentence)
print("Number of characters in the sentence :",sentence," : {}".format(number_of_characters))
if__name__=="__main__":
sentence = "Python Multiprocessing is an important library for achieving parallel programming."
start = datetime.now().microsecond
p1 = multiprocessing.Process(target=count_number_of_words,args=(sentence,))
p2 = multiprocessing.Process(target=count_number_of_characters,args=(sentence,))
p1.start()
p2.start()
p1.join()
p2.join()
end = datetime.now().microsecond
print("Time taken to execute process : {}".format((end-start)/1000),"ms")
Output:
Conclusion
In this article, we have discussed the fundamentals and types of concurrency along with the different methods to achieve it. We have looked at the syntax of all the methods along with examples. Now that we have discussed all these, it is time to put this knowledge to effect and start implementing it in your code. Grab a use case and implement the concepts and see the difference in performance.
Recommended Articles
This is a guide to Python Concurrency. Here we discuss how Does Concurrency Works and its various types, along with Examples and Code Implementation. You can also go through our other suggested articles to learn more –
