Python3 迭代器与生成器
可迭代对象与迭代器的区别、iter/next 协议、yield 生成器与生成器表达式。
迭代器
A for loop runs on the iterator protocol: iter(obj) yields an iterator, then next() is called until StopIteration. Anything implementing both methods can be looped over.
xs = [1, 2]
it = iter(xs)
print(type(it).__name__)
print(next(it), next(it))
try:
next(it)
except StopIteration:
print("StopIteration")list_iterator
1 2
StopIterationiter 与 next
next(it, default) supplies a fallback instead of raising. Iterators are one-shot: once exhausted they stay empty and do not rewind.
it = iter([1, 2, 3])
print(next(it), next(it), next(it))
print(next(it, 'done'))
xs = [1, 2, 3]
print(list(xs), list(xs))
it2 = iter(xs)
print(list(it2), list(it2))1 2 3
done
[1, 2, 3] [1, 2, 3]
[1, 2, 3] []创建迭代器
A custom class is a valid iterator once it implements __iter__ returning self and __next__ returning the next value or raising StopIteration.
class Countdown:
def __init__(self, start):
self.n = start
def __iter__(self):
return self
def __next__(self):
if self.n <= 0:
raise StopIteration
self.n -= 1
return self.n + 1
print(list(Countdown(3)))
print([x for x in Countdown(5)])[3, 2, 1]
[5, 4, 3, 2, 1]生成器
A function containing yield becomes a generator function: calling it runs no body, only returns a generator. Each next() runs to the next yield and pauses, preserving local state.
def gen():
print("start")
yield 1
print("resume")
yield 2
print("end")
g = gen()
print(type(g).__name__)
print(next(g))
print(next(g))
try:
next(g)
except StopIteration:
print("StopIteration")generator
start
1
resume
2
end
StopIteration生成器表达式
Swap the brackets of a list comprehension for parentheses and you get a generator expression, lazily evaluated. For large files or datasets it avoids holding every result in memory at once.
import sys
lst = [x * 2 for x in range(1000)]
gen = (x * 2 for x in range(1000))
print(len(lst), type(gen).__name__)
print(sys.getsizeof(lst) > sys.getsizeof(gen))
print(sum(gen))
print(sum(gen))1000 generator
True
999000
0迭代器
for 循环背后是迭代器协议：先 iter(obj) 拿到迭代器，再反复 next() 直到抛 StopIteration。任何实现了这两个方法的对象都能被 for 遍历。
xs = [1, 2]
it = iter(xs)
print(type(it).__name__)
print(next(it), next(it))
try:
next(it)
except StopIteration:
print("StopIteration")list_iterator
1 2
StopIterationiter 与 next
next(it, default) 可以给默认值，避免异常。注意迭代器是「一次性」的：遍历完就空了，再遍历不会回到开头。
it = iter([1, 2, 3])
print(next(it), next(it), next(it))
print(next(it, 'done'))
xs = [1, 2, 3]
print(list(xs), list(xs))
it2 = iter(xs)
print(list(it2), list(it2))1 2 3
done
[1, 2, 3] [1, 2, 3]
[1, 2, 3] []创建迭代器
自定义类只要实现 __iter__ 返回 self、__next__ 返回下一个值并在结束时抛 StopIteration，就是合法迭代器。
class Countdown:
def __init__(self, start):
self.n = start
def __iter__(self):
return self
def __next__(self):
if self.n <= 0:
raise StopIteration
self.n -= 1
return self.n + 1
print(list(Countdown(3)))
print([x for x in Countdown(5)])[3, 2, 1]
[5, 4, 3, 2, 1]生成器
函数里出现 yield 就变成生成器函数：调用时不执行函数体，只返回生成器对象；每次 next() 运行到下一个 yield 并暂停，局部变量状态被保留。
def gen():
print("start")
yield 1
print("resume")
yield 2
print("end")
g = gen()
print(type(g).__name__)
print(next(g))
print(next(g))
try:
next(g)
except StopIteration:
print("StopIteration")generator
start
1
resume
2
end
StopIteration生成器表达式
把列表推导式的方括号换成圆括号就是生成器表达式，惰性求值。处理大文件或大数据集时，用它可以避免一次性把结果全放进内存。
import sys
lst = [x * 2 for x in range(1000)]
gen = (x * 2 for x in range(1000))
print(len(lst), type(gen).__name__)
print(sys.getsizeof(lst) > sys.getsizeof(gen))
print(sum(gen))
print(sum(gen))1000 generator
True
999000
0