5.5. Generator Yield From¶
Since Python 3.3: PEP 380 -- Syntax for Delegating to a Subgenerator
Helps with refactoring generators
Useful for large generators which can be split into smaller ones
Delegation call
yield fromterminates onGeneratorExitfrom other functionThe value of the
yield fromexpression is the first argument to theStopIterationexception raised by the iterator when it terminatesReturn expr in a generator causes
StopIteration(expr)to be raised upon exit from the generator
>>> def run():
... yield from generator1()
... yield from generator2()
5.5.1. Example¶
>>> def run():
... for x in range(0, 3):
... yield x
... for x in range(10, 13):
... yield x
>>> def generator1():
... for x in range(0, 3):
... yield x
>>>
>>> def generator2():
... for x in range(10, 13):
... yield x
>>>
>>> def run():
... yield from generator1()
... yield from generator2()
5.5.2. Why?¶
This will not work at all! Mind that no code is executed by a function
after the return keyword.
>>> def generator1():
... for x in range(0, 3):
... yield x
>>>
>>> def generator2():
... for x in range(10, 13):
... yield x
>>>
>>> def run():
... return generator1()
... return generator2()
This will yield generators (not their values):
>>> def generator1():
... for x in range(0, 3):
... yield x
>>>
>>> def generator2():
... for x in range(10, 13):
... yield x
>>>
>>> def run():
... yield generator1()
... yield generator2()
5.5.3. Execute¶
>>> def generator1():
... for x in range(0, 3):
... yield x
>>>
>>> def generator2():
... for x in range(10, 13):
... yield x
>>>
>>> def run():
... yield from generator1()
... yield from generator2()
>>>
>>>
>>> result = run()
>>>
>>> type(result)
<class 'generator'>
>>>
>>> next(result)
0
>>> next(result)
1
>>> next(result)
2
>>> next(result)
10
>>> next(result)
11
>>> next(result)
12
>>> next(result)
Traceback (most recent call last):
StopIteration
5.5.4. Itertools Chain¶
The code is equivalent to itertools.chain():
>>> from itertools import chain
>>>
>>>
>>> def generator1():
... for x in range(0, 3):
... yield x
>>>
>>> def generator2():
... for x in range(10, 13):
... yield x
>>>
>>> def run():
... for x in chain(generator1(), generator2()):
... yield x
>>>
>>>
>>> result = run()
>>>
>>> type(result)
<class 'generator'>
>>>
>>> list(result)
[0, 1, 2, 10, 11, 12]
5.5.5. Delegation call¶
yield from turns ordinary function, into a delegation call:
>>> def worker():
... return [1, 2, 3]
>>>
>>> def run():
... yield from worker()
>>>
>>>
>>> result = run()
>>>
>>> next(result)
1
>>> next(result)
2
>>> next(result)
3
>>> next(result)
Traceback (most recent call last):
StopIteration
>>> def worker():
... return [x for x in range(0,3)]
>>>
>>> def run():
... yield from worker()
>>>
>>>
>>> result = run()
>>>
>>> next(result)
0
>>> next(result)
1
>>> next(result)
2
>>> next(result)
Traceback (most recent call last):
StopIteration
5.5.6. Yield From Sequences¶
>>> def run():
... yield from [0, 1, 2]
>>>
>>>
>>> result = run()
>>>
>>> type(result)
<class 'generator'>
>>>
>>> next(result)
0
>>> next(result)
1
>>> next(result)
2
>>> next(result)
Traceback (most recent call last):
StopIteration
5.5.7. Yield From Comprehensions¶
>>> def run():
... yield from [x for x in range(0,3)]
>>>
>>>
>>> result = run()
>>>
>>> type(result)
<class 'generator'>
>>>
>>> next(result)
0
>>> next(result)
1
>>> next(result)
2
>>> next(result)
Traceback (most recent call last):
StopIteration
5.5.8. Yield From Generator Expression¶
>>> def run():
... yield from (x for x in range(0,3))
>>>
>>>
>>> result = run()
>>>
>>> type(result)
<class 'generator'>
>>>
>>> next(result)
0
>>> next(result)
1
>>> next(result)
2
>>> next(result)
Traceback (most recent call last):
StopIteration
5.5.9. Conclusion¶
Python yield keyword creates a generator function.
It's useful when the function returns a large amount of data by splitting it into multiple chunks.
We can also send values to the generator using its
send()function.The
yield fromstatement is used to create a sub-iterator from the generator function.
5.5.10. Use Case - 0x01¶
>>> from pathlib import Path
>>>
>>>
>>> def get_files(dir):
... yield from dir.rglob('*.md')
... yield from dir.rglob('*.rst')
... yield from dir.rglob('*.txt')
>>>
>>>
>>> directory = Path.cwd()
>>>
>>> for file in get_files(directory):
... print(file)
5.5.11. Use Case - 0x02¶
>>> def get_files(directories):
... for directory in directories:
... yield from Path(directory).rglob('*.py')
>>>
>>>
>>> paths = [
... 'ChapterA/_assignments',
... 'ChapterA/_solutions',
... 'ChapterB/_solutions',
... 'ChapterB/_solutions',
... ]
>>>
>>> for file in get_files(paths):
... print(file)
5.5.12. Use Case - 0x02¶
>>> import re
>>>
>>>
>>> DATA = """
... +48 123 456 789
... +48 (12) 345-6789
... +48 12-345-6789
... +48 123 456 789
... +48 12-345-6789
... """
>>>
>>> def get_phone_number(DATA):
... yield from re.finditer('\+\d{2} \d{3} \d{3} \d{3}', DATA)
... yield from re.finditer('\+\d{2} \d{2} \d{3} \d{4}', DATA)
>>>
>>> for number in get_phone_number(DATA):
... print(number.group())
+48 123 456 789
+48 123 456 789
5.5.13. Assignments¶
"""
* Assignment: Generator YieldFrom Path
* Complexity: easy
* Lines of code: 2 lines
* Time: 3 min
English:
1. Create function `get_files()`
2. Using `Pathlib.glob()` return files in `path: Path`:
a. Python files (extension: `*.py`)
b. ReST files (extension: `*.rst`)
3. Return result as `Iterator[Path]` using `yield from`
Polish:
1. Stwórz funkcję `get_files()`
2. Używając `Pathlib.glob()` zwróć pliki w katalogu `path: Path`:
a. Pliki Python (rozszerzenie: `*.py`)
b. Pliki ReST (rozszerzenie: `*.rst`)
3. Zwróć wyniki jako `Iterator[Path]` używając `yield from`
Tests:
>>> import sys; sys.tracebacklimit = 0
>>> from inspect import isgenerator, isgeneratorfunction
>>> path = Path.cwd()
>>> file = get_files(path)
>>> assert isgeneratorfunction(get_files)
>>> assert isgenerator(file)
>>> assert isinstance(next(file), Path)
"""
from pathlib import Path
from typing import Iterator
def get_files(path: Path) -> Iterator[Path]:
...