Updated October 12, 2023
Introduction to Python Generators
Python generators are a fundamental feature for efficient and memory-friendly iteration. They enable the creation of iterable sequences without storing all values in memory simultaneously. Programmers implement generators using functions with the “yield” keyword, enabling them to pause execution and produce values one at a time. This reduces memory consumption and improves performance, especially when dealing with large data sets or infinite sequences. Generators are a key component in Python for tasks such as processing large files, streaming data, and creating custom iterators. Their simplicity and versatility make them a valuable tool for developers in various programming scenarios.
Table of Contents
Why Python Generators are important?
Python generators are important for several reasons:
- Easy Implementation: Python generators provide an easier and more concise way to create iterators compared to implementing a custom iterator class. You can create a generator function using the yield keyword, which is less verbose than defining an iterator class with __iter__ and __next__ methods.
- Memory Efficiency: Generators are memory-efficient because they produce values one at a time on the fly. Unlike a regular function that returns a list or other data structure, they do not store the entire sequence in memory, which would consume memory to hold the entire sequence. This method is beneficial when working with massive datasets or infinite sequences since it eliminates the need for excessive memory usage.
- Infinite Stream Representation: Generators suitably represent infinite data streams because they can produce values endlessly. You can use a generator to represent and work with sequences too large to fit in memory or are conceptually infinite, such as a stream of sensor data, a log file, or an infinite sequence of numbers.
- Generators Pipelining: You can connect generators in a pipeline, enabling you to create a series of operations where one generator’s output serves as the input for the next. People often refer to this as “generator pipelining” or “generator composition.” It’s a powerful method for processing data step by step, applying a series of transformations without loading the entire dataset into memory. This can be especially useful for data processing, ETL (Extract, Transform, Load) operations, and other tasks where data flows through a sequence of transformations.
How to Create Python Generators?
Creating a Python Generator/Generator Function is very simple with the help of the “yield” statement and the normal function. The yield statement is used instead of the “return” statement.
If the “yield” statement is used in a normal function, then the function is called a generator function. Yield is similar to the return function. These “yield” and “return” will provide mostly the same value from the function. The “yield” statement usually pauses the normal function by saving all of its states, and later the functions will be continued from the remaining successive calls. Generator Function may contain many yield statements as required/needed.
Syntax:
def function():
List of statements
Yield statement
Variable= sum(function)Explanation: The above syntax creates a function with some list of statements and the yield statement whenever required. Using the yield term inside of the normal function makes the function as Python Generator Function/Python Generator. Then a variable is created and assigned with the sum(function) value. One can also use the print in order to know what is the value of the sum() function’s value which is stored in the variable.
Working
It works by involving the two terms. Python Generator Function and Object. “yield” is the most important term, instead of the “return” terms in the normal function.
- Function: The generator Function of the Python Language is defined just like the normal function, but when the result needs to be produced, the term “yield” will be used instead of the “return” term to generate value. If the def function’s body contains the yield word, then the whole function becomes a Generator Function of the python programming language.
- Object: Usually, the Generator Function will return the Generator Object. The Object is used usually to call the method (next method), or it is used by placing the generator object in the “FOR IN” LOOP.
Examples
Here some of the Python programming examples are listed in order to understand what a is python generator. Check out the examples below:
Example #1
The below example is to yield/ return 5 numbers from 101 to 105. At first, a function “EasysimpleGeneratorFun1” is created using the def() function, which yields/returns the value “101” at first, then 102 for the second time, then 103 for the third time, 104 for the fourth time, and then at last 105 will be yielded at last. In order to check the above function’s yielding code using a driving code (which is nothing but the “FOR LOOP”).
Code:
def EasysimpleGeneratorFun1():
yield 101
yield 102
yield 103
yield 104
yield 105
for value01 in EasysimpleGeneratorFun1():
print(value01)Output:
Example #2
The below example of the Python generator function code is very simple in implementation. This is the program with the Python generator, which is helpful in yielding the items instead of list retuning. To implement/illustrate here made/created some function “firstnaturalsum(n1)” using the def() predefined function from the predefined library of the Python programming language. “n1” inside the “firstnaturalsum()” function represents/defines the non-negative integers which are up to the n1 value. from the value 0.
In practice, integers actually don’t even take more space for storing purposes, but if the integer is too big, the storage capacity will be somewhat less when we compare it with the normal function, which has the return function to get the result. Python programming language provides the generator functions in order to build the iterators in shortcuts. The below code is fully acceptable, but in the Python 2 version firstnaturalsum() is equivalent to xrange() (function), which is a built-in function. Likewise, in the Python 3 version, the range() function is the immutable type sequence. In general, built-in functions are always faster than all the other functions.
Code:
def firstnaturalsum(n1):
numone = 0
while numone <= n1:
yield numone
numone += 1
sum_of_first_natural = sum(firstnaturalsum(4))
print (sum_of_first_natural);Output:
Example #3
Code:
def generator_function():
a = 1
print("First value")
yield a
a += 1
print("Second value")
yield a
a += 1
print("Third value")
yield a
for x in generator_function():
print(x)Output:
Explanation: The above example is for the yield statement, which gives the series of outputs from 1 to 3. The value to return is 1, so to print this series of values, we should use a for statement to obtain the sequence of the series. By using the yield statement, we increment the value “a” by 1 and return it each time.
So the generator objects or series of values are either returned using for a loop as we did in the above example, and the other option is to use the next() method to get the series of values or generator objects.
Example #4
Code:
def simpleGeneratorFun():
yield 1
yield 2
yield 3
x = simpleGeneratorFun()
print("First value", next(x))
print("Second value", next(x))
print("Third value", next(x))Output:
Example #5
The .throw() method is used to throw exceptions with the generator.
Code:
def infinite_palindromes():
num = 0
while True:
if is_palindrome(num):
i = (yield num)
if i is not None:
num = i
num += 1
def is_palindrome(num):
# Skip single-digit inputs
if num // 10 == 0:
return False
temp = num
reversed_num = 0
while temp != 0:
reversed_num = (reversed_num * 10) + (temp % 10)
temp = temp // 10
if num == reversed_num:
return True
else:
return False
pal_gen = infinite_palindromes()
for i in pal_gen:
print(i)
digits = len(str(i))
if digits == 5:
pal_gen.throw(ValueError("We don't like large palindromes"))
pal_gen.send(10 ** (digits))
Output:
Explanation: Generator function in Python is one of the important topics in case if we are working on huge data sets or data streams or CSV files (a set of data separated by commas). Using the generator function, we can yield a series of values from huge data sets like the number of rows, etc. The below piece of code is a small example of using a generator function on data sets or large CSV files.
Code:
def csv_reader("article.txt"):
for row in open("article.txt","r"):
yield row
generator_csv = csv_reader("article.txt")
count_rows = 0
for rows in generator_csv:
count_rows+= 1
print("Count of number of rows", count_rows)
Explanation: It will Count the number of rows present in the text file.
The above example outputs the count of a number of rows from the file “article.txt”. We first open the file in reading mode and then use the yield statement in the for loop.
Recommended Articles
We hope that this EDUCBA information on “Python Generators” was beneficial to you. You can view EDUCBA’s recommended articles for more information.
