Intermediate Python

You now know the core building blocks of Python. This file covers a set of slightly more advanced — but extremely commonly used — tools that make your code more efficient, more "Pythonic," and closer to how real professional codebases are written.

1. Iterables and Iterators

What is it?

• An iterable is anything you can loop over — lists, strings, tuples, dictionaries, sets.
• An iterator is the object that actually does the "remembering where we are" while looping — produced from an iterable using iter(), and stepped through using next().

Definition: An iterable is an object capable of returning its elements one at a time; an iterator is the object that actually performs that one-at-a-time retrieval.

Simple Example

python
numbers = [10, 20, 30]   # numbers is an iterable

iterator = iter(numbers)  # create an iterator from it

print(next(iterator))   # 10
print(next(iterator))   # 20
print(next(iterator))   # 30
print(next(iterator))   # StopIteration error - nothing left!

Explanation of the Code

• iter(numbers) converts the list into an iterator object that keeps track of "where we currently are."
• Each call to next() returns the next value and moves the internal position forward.
• Once everything has been returned, calling next() again raises StopIteration — this is exactly what a for loop does automatically behind the scenes, catching that error to know when to stop.

Real-World Example

Every for loop you've ever written (for item in my_list:) works by silently calling iter() and next() behind the scenes — understanding this makes generators (next section) much easier to grasp.

Common Mistakes

• Trying to call next() on something that isn't an iterator yet (like a plain list) — you must call iter() on it first.
• Assuming all iterables are also iterators — a list is iterable, but it's not itself an iterator until you call iter() on it.

Important Points

• All iterators are iterables, but not all iterables are iterators.
• for loops handle the iter()/next()/StopIteration process automatically — you rarely need to do it manually.

2. Generators and yield

What is it?

A generator is a special kind of function that produces a sequence of values one at a time, pausing its state between each value, instead of computing and storing everything in memory at once.

Definition: A generator is a function that uses yield to produce a sequence of values lazily, one at a time, without storing them all in memory simultaneously.

Simple Example

python
def count_up_to(n):
    count = 1
    while count <= n:
        yield count
        count += 1

counter = count_up_to(5)

for number in counter:
    print(number)

Output:

1
2
3
4
5

Explanation of the Code

• yield is like return, but instead of ending the function completely, it pauses it — remembering exactly where it left off.
• Each time the generator is asked for the next value (by a for loop, or next()), it resumes right after the last yield and continues until the next yield or the function ends.
• This is fundamentally different from building a full list of [1, 2, 3, 4, 5] in memory upfront — values are produced on demand.

Why Generators Matter — Memory Efficiency

python
def generate_million_numbers():
    for i in range(1_000_000):
        yield i

# This uses very little memory, even for a million numbers,
# because values are produced one at a time, not stored all at once.
gen = generate_million_numbers()
print(next(gen))   # 0
print(next(gen))   # 1

Compare this to list(range(1_000_000)), which builds and stores all one million numbers in memory immediately.

Generator Expressions

A compact, one-line way to create a generator — looks like a list comprehension, but uses () instead of [].

python
squares_list = [x ** 2 for x in range(5)]      # list comprehension - built immediately
squares_gen = (x ** 2 for x in range(5))        # generator expression - lazy

print(squares_list)   # [0, 1, 4, 9, 16]
print(squares_gen)    # <generator object ...>  -  values not computed yet!

print(list(squares_gen))   # [0, 1, 4, 9, 16] - now they're computed

Real-World Example

Generators are ideal for reading huge files line by line, or processing large datasets, without loading everything into memory at once — a common technique in data engineering and backend systems.

Common Mistakes

• Trying to loop through a generator twice — once its values have been consumed, they're gone; you'd need to create a fresh generator to go through the sequence again.
• Confusing generator expressions (...) with tuples — (x for x in range(5)) is a generator, not a tuple.

Important Points

• yield pauses a function's execution and remembers its state, unlike return which ends it completely.
• Generators are memory-efficient — ideal for large or even infinite sequences.
• A generator can only be iterated through once.

Practice

1. Write a generator function that yields the squares of numbers from 1 to n.
2. Write a generator expression that produces all even numbers from 1 to 20, and print them using a for loop.

3. Decorators

What is it?

A decorator is a function that wraps another function, adding extra behavior before and/or after it runs — without modifying the original function's actual code.

Definition: A decorator is a function that takes another function as input and extends or modifies its behavior, returning a new function.

Simple Example

python
def my_decorator(func):
    def wrapper():
        print("Something happens before the function runs")
        func()
        print("Something happens after the function runs")
    return wrapper

@my_decorator
def say_hello():
    print("Hello!")

say_hello()

Output:

Something happens before the function runs
Hello!
Something happens after the function runs

Explanation of the Code

• @my_decorator placed above say_hello is exactly equivalent to writing say_hello = my_decorator(say_hello).
• my_decorator wraps say_hello inside wrapper(), adding extra print statements before and after calling the original function — without ever touching say_hello's own code.

A More Practical Example — Timing a Function

python
import time

def timer(func):
    def wrapper(*args, **kwargs):
        start = time.time()
        result = func(*args, **kwargs)
        end = time.time()
        print(f"{func.__name__} took {end - start:.4f} seconds")
        return result
    return wrapper

@timer
def slow_function():
    time.sleep(1)
    print("Function finished")

slow_function()

Output:

Function finished
slow_function took 1.0002 seconds

Explanation: *args, **kwargs in wrapper() lets the decorator work with any function, regardless of how many arguments it takes.

Real-World Example

Decorators are used constantly in real Python frameworks: Flask uses @app.route("/home") to turn a regular function into a web page handler; logging decorators automatically record every time a function is called.

Common Mistakes

• Forgetting to return wrapper inside the decorator, which causes the decorated function to return None instead of actually running.
• Forgetting *args, **kwargs in wrapper(), which breaks the decorator for any function that takes arguments.

Important Points

• @decorator_name above a function is shorthand for function = decorator_name(function).
• Decorators are widely used for logging, timing, authentication checks, and caching.

Practice

1. Write a decorator uppercase_result that converts a function's returned string to uppercase automatically.

4. Closures

What is it?

A closure is a function that "remembers" values from its enclosing scope, even after that outer function has finished running.

Simple Example

python
def make_multiplier(factor):
    def multiply(number):
        return number * factor   # "factor" is remembered from the outer function
    return multiply

double = make_multiplier(2)
triple = make_multiplier(3)

print(double(5))   # 10
print(triple(5))   # 15

Explanation of the Code

• make_multiplier(2) runs and finishes, returning the inner multiply function — but multiply still "remembers" that factor was 2, even though make_multiplier has already finished executing.
• This remembered value is what makes double and triple behave differently, despite both being created from the same multiply function definition.

Real-World Example

Closures are the underlying mechanism that makes decorators work — the wrapper() function in a decorator "closes over" the original function it's wrapping.

Common Mistakes

• Assuming the outer function's variables are lost once it returns — with closures, they're actually preserved for the inner function's use.

Important Points

• A closure "closes over" variables from its enclosing scope, keeping them alive as long as the inner function might still need them.
• Closures are a key concept behind decorators and many functional-programming patterns in Python.

Practice

1. Write a closure-based function make_greeting(greeting) that returns a function which greets any name passed to it with that greeting.

5. Context Managers (with)

What is it?

You've already used context managers with with open(...) as file: in the File Handling section. A context manager is any object that defines proper "setup" and "cleanup" behavior automatically, used with the with statement.

Creating Your Own Context Manager

python
class ManagedFile:
    def __init__(self, filename, mode):
        self.filename = filename
        self.mode = mode

    def __enter__(self):
        self.file = open(self.filename, self.mode)
        return self.file

    def __exit__(self, exc_type, exc_value, traceback):
        self.file.close()

with ManagedFile("notes.txt", "w") as file:
    file.write("Hello from a custom context manager!")

Explanation of the Code

• __enter__ runs when entering the with block — it sets things up (here, opening the file) and returns the object to use inside the block.
• __exit__ runs automatically when the block ends, whether it finished normally or an error occurred — it handles cleanup (here, closing the file).

Real-World Example

Context managers are used for anything requiring guaranteed setup/cleanup: file handling, database connections, network sockets, and thread locks.

Important Points

• Any class implementing both __enter__ and __exit__ can be used with with.
• Python's built-in open() already works this way — that's why with open(...) as f: works.

6. The property Decorator

What is it?

property lets you define a method that behaves like a regular attribute — accessed without parentheses — while still letting you run custom logic (like validation) behind the scenes.

Simple Example

python
class Circle:
    def __init__(self, radius):
        self._radius = radius

    @property
    def area(self):
        return 3.14159 * self._radius ** 2

    @property
    def radius(self):
        return self._radius

    @radius.setter
    def radius(self, value):
        if value < 0:
            raise ValueError("Radius cannot be negative")
        self._radius = value

circle = Circle(5)
print(circle.area)     # 78.53975 - called like an attribute, not area()

circle.radius = 10     # uses the setter, runs validation
print(circle.area)     # 314.159

Explanation of the Code

• @property above area means circle.area runs the method automatically — no parentheses needed, even though it's really a method underneath.
• @radius.setter lets you control what happens when someone tries to assign a new value to circle.radius — here, rejecting negative values.

Real-World Example

Properties are commonly used to expose "calculated" attributes (like area, computed from radius) or to add validation when a value is set, without breaking the simple object.attribute syntax users expect.

Important Points

• @property makes a method accessible like an attribute (read-only, unless paired with a setter).
• @attributename.setter allows controlled, validated assignment to that "property."

Practice

1. Create a Temperature class with a celsius property and a fahrenheit property (calculated automatically from celsius).

7. Useful Built-in Functions

enumerate() — Get Index and Value Together

python
fruits = ["apple", "banana", "cherry"]

for index, fruit in enumerate(fruits):
    print(index, fruit)

Output:

0 apple
1 banana
2 cherry

Real-World Use: Whenever you need both the position and the value while looping — much cleaner than manually tracking a separate counter variable.

zip() — Combine Multiple Sequences Together

python
names = ["Aditi", "Rohan", "Zara"]
marks = [85, 90, 78]

for name, mark in zip(names, marks):
    print(f"{name}: {mark}")

Output:

Aditi: 85
Rohan: 90
Zara: 78

Real-World Use: Combining related lists — like names and marks, or product names and prices — that need to be processed together.

any() and all()

python
numbers = [2, 4, 6, 7, 8]

print(any(n % 2 != 0 for n in numbers))   # True  - at least one odd number
print(all(n % 2 == 0 for n in numbers))   # False - not every number is even

Explanation: any() returns True if at least one item satisfies the condition. all() returns True only if every item satisfies it.

sorted(), min(), max()

python
students = [("Aditi", 85), ("Rohan", 92), ("Zara", 78)]

print(sorted(students, key=lambda s: s[1]))              # sort by marks, ascending
print(sorted(students, key=lambda s: s[1], reverse=True)) # descending
print(max(students, key=lambda s: s[1]))                  # highest scorer
print(min(students, key=lambda s: s[1]))                  # lowest scorer

Output:

[('Zara', 78), ('Aditi', 85), ('Rohan', 92)]
[('Rohan', 92), ('Aditi', 85), ('Zara', 78)]
('Rohan', 92)
('Zara', 78)

Explanation: The key parameter tells Python what to sort/compare by — here, the second item in each tuple (the marks), rather than comparing the tuples directly.

Comparison Table — Key Built-ins

Common Mistakes

• Forgetting sorted() returns a new list — it doesn't sort the original list in place (use .sort() for that, on lists specifically).
• Forgetting the key parameter when sorting complex data like tuples or objects by a specific field.

Practice

1. Use enumerate() to print each item in a list along with its position, starting the count from 1 instead of 0 (hint: enumerate(list, start=1)).
2. Use zip() to combine a list of product names and a list of prices into a dictionary.
3. Use sorted() with a key to sort a list of dictionaries (each with a "score" key) by score, highest first.

Common Beginner Mistakes — Summary for This Section

• Confusing iterables with iterators.
• Trying to loop through an already-exhausted generator a second time.
• Forgetting return wrapper inside a decorator.
• Forgetting *args, **kwargs in a decorator's inner function.
• Forgetting sorted() returns a new list rather than sorting in place.

Cheat Sheet — Intermediate Python

python
# Generators
def gen():
    yield 1
    yield 2

# Decorators
def decorator(func):
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)
    return wrapper

@decorator
def my_func():
    pass

# property
class C:
    @property
    def value(self):
        return self._value

    @value.setter
    def value(self, v):
        self._value = v

# Useful built-ins
enumerate(list, start=0)
zip(list1, list2)
any(condition for x in list)
all(condition for x in list)
sorted(list, key=lambda x: x[1], reverse=True)
max(list, key=lambda x: x[1])
min(list, key=lambda x: x[1])

Interview Questions

Q1. What is the difference between an iterable and an iterator? Answer: An iterable is anything you can loop over (like a list). An iterator is the object produced by calling iter() on an iterable, which actually tracks position and produces values one at a time via next().

Q2. What is the difference between `return` and `yield`? Answer: return ends a function completely and sends back one value. yield pauses the function, remembers its state, and can produce multiple values over time as it's iterated.

Q3. Why are generators more memory-efficient than lists? Answer: Generators produce values one at a time, on demand, instead of computing and storing the entire sequence in memory upfront.

Q4. What is a decorator, and what does the `@decorator_name` syntax actually mean? Answer: A decorator is a function that wraps another function to add extra behavior. @decorator_name above a function is shorthand for function = decorator_name(function).

Q5. What is a closure? Answer: A function that remembers and has access to variables from its enclosing scope, even after the outer function has finished executing.

Q6. What does the `@property` decorator do? Answer: It lets a method be accessed like a regular attribute (without parentheses), while still allowing custom logic — often used for computed values or validated attribute assignment (via a paired setter).

Q7. What is the difference between `any()` and `all()`? Answer: any() returns True if at least one element satisfies the condition. all() returns True only if every element satisfies it.

Practice Questions

Beginner

1. Write a generator function that yields numbers from 1 to 10.
2. Use enumerate() to print each character of a string along with its index.
3. Use zip() to combine two lists of equal length into a list of tuples.
4. Use any() to check if a list contains at least one negative number.
5. Use sorted() to sort a list of numbers in descending order.

Intermediate

1. Write a decorator that prints "Function is starting..." before a function runs and "Function has finished." after it runs.
2. Write a generator expression that yields the square of every odd number from 1 to 20.
3. Create a class with a property for fahrenheit, calculated automatically from a stored celsius value.
4. Write a closure-based function make_counter() that returns a function which increases and returns a count every time it's called.
5. Use sorted() with a key and lambda to sort a list of student dictionaries by their "marks" value.

Challenge

1. Write a custom context manager (using a class with __enter__/__exit__) that measures and prints how long the code inside the with block took to run.
2. Write a decorator @retry that automatically retries a function up to 3 times if it raises an exception, before finally letting the exception propagate.
3. Write a generator function that produces an infinite sequence of Fibonacci numbers, and use it with a loop that stops after printing the first 15 values.