Concurrency & Performance

Some programs need to do multiple things at once — downloading several files simultaneously, handling many web requests, or crunching heavy calculations faster by using multiple CPU cores. This file covers Python's tools for concurrency (threading, multiprocessing, asyncio) and for measuring and improving performance.

1. Concurrency vs Parallelism

What is it?

• Concurrency means managing multiple tasks that are in progress at overlapping times — not necessarily running at the exact same instant, but making progress on several things by switching between them.
• Parallelism means multiple tasks genuinely running at the same physical instant, using multiple CPU cores.

Real-World Analogy

A single chef juggling three dishes at once (stirring one, chopping for another, checking the oven for a third) is concurrency — one chef, switching between tasks. Three chefs each cooking their own dish simultaneously is parallelism — truly simultaneous work.

Important Points

• Concurrency is about structure (dealing with many things), parallelism is about execution (doing many things at the exact same time).
• Python's approach to each — threading, multiprocessing, asyncio — has important trade-offs, covered below.

2. Processes vs Threads

What is it?

• A process is an independent running program, with its own separate memory space.
• A thread is a smaller unit of execution within a process — multiple threads in the same process share the same memory.

Comparison Table

3. The Global Interpreter Lock (GIL)

What is it?

The GIL is a mechanism in CPython (introduced in the Advanced Python file) that allows only one thread to execute Python bytecode at any given moment, even on a multi-core machine.

Definition: The GIL (Global Interpreter Lock) is a lock in CPython that ensures only one thread executes Python bytecode at a time, limiting true parallel execution of Python code across threads.

Why Does This Matter?

Because of the GIL, using multiple threads for CPU-heavy Python code (like complex calculations) generally does not make it run faster on multiple cores — the GIL prevents true parallel execution of Python bytecode. However, threads are still very useful for I/O-bound tasks (waiting on files, network requests), since the GIL is released while a thread is waiting on I/O.

Important Points

• The GIL is specific to CPython (the standard Python implementation) — not all Python implementations have this limitation.
• For genuinely parallel CPU-heavy work, multiprocessing (separate processes, each with its own GIL) is the standard Python solution, not threading.

4. Multithreading

What is it?

Python's threading module lets you run multiple threads within a single process — well-suited for I/O-bound tasks (waiting on network responses, file reads, database queries), where a thread mostly waits rather than computes.

Simple Example

python
import threading
import time

def download_file(name):
    print(f"Starting download: {name}")
    time.sleep(2)   # simulates waiting on a network request
    print(f"Finished download: {name}")

start = time.time()

threads = []
for filename in ["file1.zip", "file2.zip", "file3.zip"]:
    thread = threading.Thread(target=download_file, args=(filename,))
    threads.append(thread)
    thread.start()

for thread in threads:
    thread.join()

print(f"Total time: {time.time() - start:.2f} seconds")

Output (approximately):

Starting download: file1.zip
Starting download: file2.zip
Starting download: file3.zip
Finished download: file1.zip
Finished download: file2.zip
Finished download: file3.zip
Total time: 2.01 seconds

Explanation of the Code

• All three "downloads" start almost immediately and run concurrently — the total time is about 2 seconds (the time for one download), not 6 seconds (which running them one after another would take).
• thread.start() begins running the thread; thread.join() tells the main program to wait until that thread has finished before continuing.
• This works well here specifically because time.sleep() simulates waiting (I/O), during which the GIL is released, letting other threads make progress.

Common Mistakes

• Using threading for CPU-heavy calculations, expecting a speedup — the GIL prevents genuine parallel execution of pure Python computation across threads.
• Forgetting .join(), causing the main program to potentially finish before background threads complete their work.

Important Points

• Threading is well-suited for I/O-bound tasks: network requests, file operations, waiting on external resources.
• Threading is generally not effective for CPU-bound tasks in CPython, due to the GIL.

5. Multiprocessing

What is it?

Python's multiprocessing module runs code in separate processes, each with its own Python interpreter and memory space (and therefore, its own GIL) — enabling genuine parallel execution across multiple CPU cores, ideal for CPU-heavy tasks.

Simple Example

python
import multiprocessing
import time

def calculate_squares(numbers):
    return [n ** 2 for n in numbers]

if __name__ == "__main__":
    numbers = list(range(1, 10_000_000))
    half = len(numbers) // 2

    start = time.time()

    with multiprocessing.Pool(processes=2) as pool:
        results = pool.map(calculate_squares, [numbers[:half], numbers[half:]])

    print(f"Total time: {time.time() - start:.2f} seconds")

Explanation of the Code

• multiprocessing.Pool(processes=2) creates 2 separate worker processes.
• .map() splits the work (here, two halves of the number list) across the available processes, running them genuinely in parallel on separate CPU cores.
• The if __name__ == "__main__": guard (from the Modules & Packages file) is required for multiprocessing code on some platforms, to prevent child processes from accidentally re-running the entire script from scratch.

Common Mistakes

• Forgetting the if __name__ == "__main__": guard when using multiprocessing, which can cause runaway process creation or errors, especially on Windows.
• Using multiprocessing for small, lightweight tasks — the overhead of creating separate processes can outweigh any benefit for trivial work.

Important Points

• Multiprocessing achieves genuine parallelism by using entirely separate processes, each with its own GIL.
• Best suited for CPU-bound tasks: heavy calculations, data processing, image/video processing.

Comparison Table — Threading vs Multiprocessing vs asyncio

6. asyncio — Asynchronous Programming

What is it?

asyncio allows a single thread to efficiently handle many I/O-bound tasks by voluntarily "pausing" a task while it's waiting (e.g., for a network response) and working on something else in the meantime — without the overhead of actual threads or processes.

Definition: asyncio is Python's library for writing concurrent code using the async/await syntax, allowing a single thread to efficiently manage many I/O-bound tasks.

async and await

python
import asyncio

async def download_file(name):
    print(f"Starting download: {name}")
    await asyncio.sleep(2)   # non-blocking "wait" - lets other tasks run meanwhile
    print(f"Finished download: {name}")

async def main():
    await asyncio.gather(
        download_file("file1.zip"),
        download_file("file2.zip"),
        download_file("file3.zip")
    )

asyncio.run(main())

Output (approximately):

Starting download: file1.zip
Starting download: file2.zip
Starting download: file3.zip
Finished download: file1.zip
Finished download: file2.zip
Finished download: file3.zip

(Total time: about 2 seconds — all three "downloads" progress concurrently.)

Explanation of the Code

• async def marks a function as a coroutine — a special function that can be paused and resumed.
• await asyncio.sleep(2) pauses this specific task, letting the event loop run other pending tasks during the wait, then resumes once the wait is over.
• asyncio.gather(...) runs multiple coroutines concurrently, waiting for all of them to finish.
• asyncio.run(main()) starts the "event loop" — the mechanism that manages switching between tasks — and runs the main coroutine to completion.

Real-World Example

asyncio is heavily used in modern web frameworks (like FastAPI, covered in the Web Development file) and network applications needing to handle thousands of simultaneous connections efficiently, without the overhead of one thread per connection.

Common Mistakes

• Forgetting await before an async function call, which returns a coroutine object rather than actually running it.
• Mixing regular blocking code (like time.sleep()) inside async functions instead of the async-aware equivalent (asyncio.sleep()), which would block the entire event loop, defeating the purpose.
• Assuming asyncio provides true parallelism for CPU-heavy work — it doesn't; it's specifically for efficiently handling many I/O-bound waits on a single thread.

Important Points

• asyncio shines for high-volume I/O-bound work (many simultaneous network requests) with very low overhead.
• Every I/O operation inside an async function needs an async-aware version (asyncio.sleep() instead of time.sleep(), async HTTP libraries instead of requests, etc.) to actually benefit from asyncio's efficiency.

Practice

1. Write an asyncio program that "fetches" 5 items concurrently, each simulated with asyncio.sleep(1), and confirm the total runtime is close to 1 second rather than 5.

7. Profiling

What is it?

Profiling measures exactly where your program spends its time, helping you identify actual performance bottlenecks — rather than guessing which part of the code is slow.

Simple Example — timeit

python
import timeit

def using_loop():
    result = []
    for i in range(1000):
        result.append(i ** 2)
    return result

def using_comprehension():
    return [i ** 2 for i in range(1000)]

loop_time = timeit.timeit(using_loop, number=1000)
comprehension_time = timeit.timeit(using_comprehension, number=1000)

print(f"Loop: {loop_time:.4f}s")
print(f"Comprehension: {comprehension_time:.4f}s")

Explanation: timeit runs a piece of code many times (here, 1000 times) and measures the total time, giving a reliable comparison between two approaches — useful for confirming whether an "optimization" actually helps.

Simple Example — cProfile

python
import cProfile

def slow_function():
    total = 0
    for i in range(1_000_000):
        total += i
    return total

cProfile.run("slow_function()")

Explanation: cProfile gives a detailed breakdown of how many times each function was called and how much total time was spent inside it — invaluable for pinpointing exactly which function is the actual bottleneck in a larger program, rather than guessing.

Important Points

• Always profile before optimizing — intuition about what's "slow" is often wrong, and optimizing the wrong part wastes effort.
• timeit is best for comparing small snippets; cProfile is best for analyzing a larger program's overall behavior.

8. Optimization Tips

General Guidance

• Measure first — use profiling tools before assuming where the bottleneck is.
• Use built-in functions and libraries — they're typically implemented in C and much faster than equivalent hand-written Python loops.
• Use list comprehensions / generator expressions where appropriate — often faster and more readable than manual loops.
• Avoid unnecessary work inside loops — move calculations that don't change out of the loop body.
• Use appropriate data structures — e.g., a set for membership testing (in) is far faster than a list for large collections.

Simple Example

python
# Slower: checking membership in a list
big_list = list(range(100_000))
print(99_999 in big_list)   # scans through the list one item at a time

# Faster: checking membership in a set
big_set = set(range(100_000))
print(99_999 in big_set)    # near-instant lookup

Explanation: Sets use a hash-based lookup internally, making membership checks dramatically faster than scanning through a list, especially as the collection grows large.

9. Caching

What is it?

Caching stores the result of an expensive computation so it doesn't need to be recalculated the next time the same input occurs — trading a bit of memory for a significant speed improvement on repeated calls.

functools.lru_cache (Recap and Deeper Look)

python
from functools import lru_cache
import time

@lru_cache(maxsize=None)
def slow_calculation(n):
    time.sleep(1)   # simulate an expensive operation
    return n * n

print(slow_calculation(5))   # takes about 1 second (first call)
print(slow_calculation(5))   # instant! (cached result from before)

Explanation: @lru_cache automatically remembers the result for each unique set of arguments — calling slow_calculation(5) a second time returns the cached answer instantly, skipping the expensive computation entirely.

Important Points

• maxsize=None means the cache can grow without limit; setting a specific number limits how many recent results are kept, evicting the "Least Recently Used" ones once full.
• Caching is most effective for functions that are called repeatedly with the same arguments and have no side effects (their output depends only on their input).

Common Beginner Mistakes — Summary for This Section

• Using threading for CPU-heavy work, expecting a speedup that the GIL prevents.
• Forgetting if __name__ == "__main__": with multiprocessing.
• Using blocking calls (time.sleep()) instead of async-aware equivalents inside asyncio code.
• Optimizing code without first profiling to confirm where the actual bottleneck is.

Cheat Sheet — Concurrency & Performance

python
# Threading (I/O-bound)
import threading
t = threading.Thread(target=func, args=(...,))
t.start(); t.join()

# Multiprocessing (CPU-bound)
import multiprocessing
if __name__ == "__main__":
    with multiprocessing.Pool(processes=4) as pool:
        results = pool.map(func, data)

# asyncio (I/O-bound, single-threaded)
import asyncio
async def task():
    await asyncio.sleep(1)
asyncio.run(asyncio.gather(task(), task()))

# Profiling
import timeit, cProfile
timeit.timeit(func, number=1000)
cProfile.run("func()")

# Caching
from functools import lru_cache
@lru_cache(maxsize=None)
def expensive_func(n): ...

Interview Questions

Q1. What is the GIL, and how does it affect multithreading in Python? Answer: The Global Interpreter Lock allows only one thread to execute Python bytecode at a time in CPython, preventing true parallel execution of CPU-bound Python code across threads — though threads remain useful for I/O-bound tasks, since the GIL is released during I/O waits.

Q2. When would you use multiprocessing instead of threading? Answer: For CPU-bound tasks (heavy calculations), since multiprocessing uses separate processes (each with its own GIL), achieving genuine parallelism across multiple CPU cores — something threading cannot do due to the GIL.

Q3. What is the difference between concurrency and parallelism? Answer: Concurrency means managing multiple tasks with overlapping progress, potentially by switching between them. Parallelism means multiple tasks genuinely executing at the exact same instant, typically across multiple CPU cores.

Q4. What is `asyncio` best suited for? Answer: Efficiently handling many I/O-bound tasks (like network requests) concurrently on a single thread, with very low overhead — not suited for CPU-bound work, since it doesn't provide true parallelism.

Q5. Why should you profile code before optimizing it? Answer: Intuition about which part of a program is slow is often wrong; profiling tools (like cProfile) reveal the actual bottleneck, ensuring optimization effort is spent where it will genuinely help.

Q6. What does `functools.lru_cache` do? Answer: It automatically caches a function's results based on its arguments, returning the cached result instantly on repeated calls with the same arguments instead of recomputing them.

Practice Questions

Beginner

1. Write a threading example that "downloads" 3 files concurrently using time.sleep() to simulate the wait.
2. Use timeit to compare the speed of building a list with a loop versus a list comprehension.
3. Add @lru_cache to a slow recursive Fibonacci function and observe the speed difference for larger inputs.
4. Write a simple asyncio program with one coroutine that sleeps for 1 second and prints a message.
5. Explain, in your own words, the difference between a process and a thread.

Intermediate

1. Use multiprocessing.Pool to calculate squares of a large list of numbers, splitting the work across multiple processes.
2. Use asyncio.gather() to run 5 simulated "API calls" concurrently, each taking 1 second, and confirm the total time is close to 1 second.
3. Use cProfile to profile a function that performs a moderately expensive calculation, and identify which part takes the most time.
4. Compare membership testing speed (in) between a large list and a large set of the same size using timeit.
5. Explain (in your own words) why using threads for a CPU-heavy calculation likely won't provide a speedup in CPython.

Challenge

1. Build a small "web scraper" simulation using asyncio that "fetches" 10 pages concurrently (each simulated with asyncio.sleep()), and compare its total time against doing the same 10 fetches sequentially.
2. Use multiprocessing to parallelize a CPU-heavy task (like checking primality for a large range of numbers) across 4 processes, and measure the speedup compared to a single-process version.
3. Profile a small program with multiple functions using cProfile, identify the single biggest bottleneck, optimize it (e.g., with caching or a better data structure), and re-profile to confirm the improvement.