Debug and Profile Python Apps from the CLI

Intermediate12 min

Use Python's built-in debugger (pdb), profiling tools (cProfile), and memory analysis to find bugs and performance bottlenecks from the command line.

Prerequisites

-Python 3.8+ installed

Steps

Run a script with the debugger

Start the script under the Python debugger. Execution pauses at the first line, letting you step through code, set breakpoints, and inspect variables.

$ python -m pdb script.py

Use breakpoint() in your code to set inline breakpoints instead of running the entire script under pdb.

Run with post-mortem debugging on crash

If a script crashes, use pdb.pm() to open the debugger at the point of the exception. You can inspect the traceback, local variables, and call stack.

$ python -c "import pdb; import script; pdb.pm()"

The simpler approach is to add 'breakpoint()' on the line before the crash and run normally, which drops you into pdb at that exact location.

Profile a script with cProfile

Run the script with the built-in profiler and sort results by cumulative time. This shows which functions consume the most total time including their callees.

$ python -m cProfile -s cumulative script.py

Save profiling data and analyze it

Save profiling data to a binary file and open the interactive pstats browser. You can sort, filter, and drill into call trees.

$ python -m cProfile -o profile.prof script.py && python -m pstats profile.prof

Use 'snakeviz profile.prof' for a visual flame graph in your browser. Install with pip install snakeviz.

Measure memory usage with tracemalloc

Trace memory allocations and display the top 10 lines by memory consumption. This helps identify memory leaks and excessive allocations.

$ python -c "import tracemalloc; tracemalloc.start(); exec(open('script.py').read()); snapshot = tracemalloc.take_snapshot(); [print(s) for s in snapshot.statistics('lineno')[:10]]"

Time a specific code snippet

Benchmark a code snippet by running it 1000 times and reporting the average execution time. The -s flag runs setup code once before timing begins.

$ python -m timeit -n 1000 -s 'data = list(range(10000))' 'sorted(data)'

Full Script

full-script.sh

#!/bin/bash
# Debug and profiling workflow
set -e

# Create a sample script to debug/profile
cat > sample_script.py << 'PYEOF'
import time

def slow_function(n):
    """Deliberately slow function for profiling demo."""
    total = 0
    for i in range(n):
        total += sum(range(i))
    return total

def fast_function(n):
    """Optimized version using math."""
    return sum(i * (i - 1) // 2 for i in range(n))

if __name__ == "__main__":
    n = 500
    print(f"Running slow_function({n})...")
    result1 = slow_function(n)
    print(f"Result: {result1}")

    print(f"Running fast_function({n})...")
    result2 = fast_function(n)
    print(f"Result: {result2}")

    assert result1 == result2, "Results should match"
    print("Both functions produce the same result.")
PYEOF

echo "=== Profiling with cProfile ==="
python -m cProfile -s cumulative sample_script.py

echo ""
echo "=== Timing comparison ==="
python -m timeit -n 100 -s 'from sample_script import slow_function' 'slow_function(100)'
python -m timeit -n 100 -s 'from sample_script import fast_function' 'fast_function(100)'

echo ""
echo "Profiling complete. Review the output above to identify bottlenecks."

FAQ

Discussion

Loading comments...

#!/bin/bash # Debug and profiling workflow set -e # Create a sample script to debug/profile cat > sample_script.py << 'PYEOF' import time def slow_function(n): """Deliberately slow function for profiling demo.""" total = 0 for i in range(n): total += sum(range(i)) return total def fast_function(n): """Optimized version using math.""" return sum(i * (i - 1) // 2 for i in range(n)) if __name__ == "__main__": n = 500 print(f"Running slow_function({n})...") result1 = slow_function(n) print(f"Result: {result1}") print(f"Running fast_function({n})...") result2 = fast_function(n) print(f"Result: {result2}") assert result1 == result2, "Results should match" print("Both functions produce the same result.") PYEOF echo "=== Profiling with cProfile ===" python -m cProfile -s cumulative sample_script.py echo "" echo "=== Timing comparison ===" python -m timeit -n 100 -s 'from sample_script import slow_function' 'slow_function(100)' python -m timeit -n 100 -s 'from sample_script import fast_function' 'fast_function(100)' echo "" echo "Profiling complete. Review the output above to identify bottlenecks."