Learn how iteration works under the hood in Python — iterables, iterators, and the iterator protocol.

Iterators

You have been using iteration since the beginning — for loops, list(), zip(), enumerate(). But how does it actually work? What happens when Python runs for item in something?

The answer is the iterator protocol — a simple contract that any object can implement to make itself iterable.

Iterables vs Iterators

These two words sound similar but mean different things:

Iterable — any object you can loop over. Lists, strings, tuples, dictionaries, sets, files, ranges. An iterable knows how to give you an iterator.

Iterator — the object that actually does the walking. It remembers where it is and knows how to get the next item.

# a list is iterable — you can loop over it
numbers = [1, 2, 3]

# iter() gives you an iterator from the iterable
iterator = iter(numbers)

# next() gets the next item from the iterator
print(next(iterator))   # 1
print(next(iterator))   # 2
print(next(iterator))   # 3
print(next(iterator))   # StopIteration — no more items

Every for loop is secretly doing exactly this.

How a for loop really works

When Python sees for item in numbers, it:

Calls iter(numbers) to get an iterator
Calls next() on the iterator repeatedly
Each call returns the next value and assigns it to item
When next() raises StopIteration, the loop ends

This is the whole mechanism. Everything that works with for implements this protocol.

The iterator protocol

Two methods make an object an iterator:

__iter__() — returns the iterator object itself
__next__() — returns the next value, raises StopIteration when done

Any object that implements both is a valid iterator.

Building a custom iterator

Let's build a counter — an iterator that counts from a start value to an end value:

class Counter:
    def __init__(self, start, end):
        self.current = start
        self.end = end

    def __iter__(self):
        return self   # the iterator is the object itself

    def __next__(self):
        if self.current > self.end:
            raise StopIteration
        value = self.current
        self.current += 1
        return value


counter = Counter(1, 5)

for number in counter:
    print(number)

Output:

You can also use it manually with next():

counter = Counter(1, 3)

print(next(counter))   # 1
print(next(counter))   # 2
print(next(counter))   # 3
print(next(counter))   # StopIteration

A real custom iterator — file reader

An iterator that reads a file and yields only non-empty, stripped lines:

class CleanLineReader:
    def __init__(self, filepath):
        self.filepath = filepath
        self.file = None
        self.lines = None
        self.index = 0

    def __iter__(self):
        self.file = open(self.filepath, "r", encoding="utf-8")
        self.lines = self.file.readlines()
        self.index = 0
        return self

    def __next__(self):
        while self.index < len(self.lines):
            line = self.lines[self.index].strip()
            self.index += 1
            if line:             # skip empty lines
                return line
        self.file.close()
        raise StopIteration


for line in CleanLineReader("notes.txt"):
    print(line)

Iterable vs Iterator — the key difference

An iterable gives you a fresh iterator every time you call iter() on it. You can loop over it multiple times:

numbers = [1, 2, 3]   # iterable

for n in numbers:
    print(n)   # 1 2 3

for n in numbers:
    print(n)   # 1 2 3 — works again, fresh iterator each time

An iterator is consumed — once exhausted, it is done:

iterator = iter([1, 2, 3])   # iterator

for n in iterator:
    print(n)   # 1 2 3

for n in iterator:
    print(n)   # nothing — already exhausted

This catches many people off guard. If you store an iterator and try to loop over it twice, the second loop does nothing. Always call iter() again or use the original iterable if you need to loop multiple times.

Making a separate iterable and iterator

The cleaner pattern for reusable iterables is to separate the iterable from the iterator. The iterable creates a fresh iterator object each time:

class CounterIterator:
    """The iterator — does the walking."""
    def __init__(self, start, end):
        self.current = start
        self.end = end

    def __iter__(self):
        return self

    def __next__(self):
        if self.current > self.end:
            raise StopIteration
        value = self.current
        self.current += 1
        return value


class Counter:
    """The iterable — creates a fresh iterator each time."""
    def __init__(self, start, end):
        self.start = start
        self.end = end

    def __iter__(self):
        return CounterIterator(self.start, self.end)


counter = Counter(1, 3)

for n in counter:
    print(n)   # 1 2 3

for n in counter:
    print(n)   # 1 2 3 — works again, fresh iterator

Built-in functions that work with iterators

Because Python's built-in functions all use the iterator protocol, they work with any iterable — including your custom ones:

counter = Counter(1, 10)

print(list(counter))      # [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
print(sum(counter))       # 55
print(max(counter))       # 10
print(min(counter))       # 1
print(list(zip(counter, counter)))   # careful — same iterator!

counter = Counter(1, 5)

# enumerate works too
for index, value in enumerate(counter):
    print(f"{index}: {value}")

Output:

0: 1
1: 2
2: 3
3: 4
4: 5

iter() with a sentinel

There is a second form of iter() — iter(callable, sentinel). It calls the callable repeatedly until it returns the sentinel value, then stops:

import random

# keep rolling a dice until you get a 6
rolls = list(iter(lambda: random.randint(1, 6), 6))
print(rolls)   # [3, 1, 4, 1, 5] — whatever came before the 6

A practical use — read a file in chunks:

with open("large_file.txt", "r") as f:
    for chunk in iter(lambda: f.read(1024), ""):
        process(chunk)

Reads 1024 characters at a time until the file is empty.

itertools — powerful iterator tools

Python's itertools module gives you ready-made iterators for common patterns:

import itertools

chain() — combine multiple iterables into one:

a = [1, 2, 3]
b = [4, 5, 6]
c = [7, 8, 9]

for n in itertools.chain(a, b, c):
    print(n, end=" ")
# 1 2 3 4 5 6 7 8 9

islice() — slice an iterator:

counter = Counter(1, 100)

# get the first 5 items without consuming the rest
first_five = list(itertools.islice(counter, 5))
print(first_five)   # [1, 2, 3, 4, 5]

cycle() — repeat an iterable forever:

colors = itertools.cycle(["red", "green", "blue"])

for i, color in zip(range(7), colors):
    print(f"Item {i}: {color}")

Output:

Item 0: red
Item 1: green
Item 2: blue
Item 3: red
Item 4: green
Item 5: blue
Item 6: red

count() — infinite counter:

for n in itertools.count(start=10, step=5):
    if n > 30:
        break
    print(n, end=" ")
# 10 15 20 25 30

product() — cartesian product:

sizes = ["S", "M", "L"]
colors = ["red", "blue"]

for size, color in itertools.product(sizes, colors):
    print(f"{size}-{color}")

Output:

S-red
S-blue
M-red
M-blue
L-red
L-blue

groupby() — group consecutive items:

data = [
    {"name": "Ali",    "dept": "Engineering"},
    {"name": "Sara",   "dept": "Engineering"},
    {"name": "Omar",   "dept": "Marketing"},
    {"name": "Fatima", "dept": "Marketing"},
    {"name": "Zainab", "dept": "Engineering"},
]

# must be sorted by the key first
data.sort(key=lambda x: x["dept"])

for dept, members in itertools.groupby(data, key=lambda x: x["dept"]):
    names = [m["name"] for m in members]
    print(f"{dept}: {names}")

Output:

Engineering: ['Ali', 'Sara', 'Zainab']
Marketing: ['Omar', 'Fatima']

A real example — paginated data iterator

An iterator that simulates fetching paginated data — one page at a time:

class PaginatedData:
    """
    Iterates over paginated data one page at a time.
    Simulates an API that returns data in pages.
    """
    def __init__(self, data, page_size=3):
        self.data = data
        self.page_size = page_size
        self.page = 0

    def __iter__(self):
        self.page = 0
        return self

    def __next__(self):
        start = self.page * self.page_size
        end = start + self.page_size

        if start >= len(self.data):
            raise StopIteration

        self.page += 1
        return self.data[start:end]


users = ["Ali", "Sara", "Omar", "Fatima", "Zainab", "Ahmed", "Hassan"]
pages = PaginatedData(users, page_size=3)

for page_number, page in enumerate(pages, start=1):
    print(f"Page {page_number}: {page}")

Output:

Page 1: ['Ali', 'Sara', 'Omar']
Page 2: ['Fatima', 'Zainab', 'Ahmed']
Page 3: ['Hassan']

Summary

Concept	Meaning
Iterable	Object you can loop over — has `__iter__()`
Iterator	Object that does the walking — has `__iter__()` and `__next__()`
`iter(x)`	Get an iterator from an iterable
`next(x)`	Get the next value from an iterator
`StopIteration`	Signal that the iterator is exhausted
Iterable vs Iterator	Iterable can be looped multiple times — iterator is consumed once

Iterators

On this page