1---
2name: python-performance-optimization
3description: Profile and optimize Python code using cProfile, memory profilers, and performance best practices. Use when debugging slow Python code, optimizing bottlenecks, or improving application performance.
4---
5 
6# Python Performance Optimization
7 
8Comprehensive guide to profiling, analyzing, and optimizing Python code for better performance, including CPU profiling, memory optimization, and implementation best practices.
9 
10## When to Use This Skill
11 
12- Identifying performance bottlenecks in Python applications
13- Reducing application latency and response times
14- Optimizing CPU-intensive operations
15- Reducing memory consumption and memory leaks
16- Improving database query performance
17- Optimizing I/O operations
18- Speeding up data processing pipelines
19- Implementing high-performance algorithms
20- Profiling production applications
21 
22## Core Concepts
23 
24### 1. Profiling Types
25 
26- **CPU Profiling**: Identify time-consuming functions
27- **Memory Profiling**: Track memory allocation and leaks
28- **Line Profiling**: Profile at line-by-line granularity
29- **Call Graph**: Visualize function call relationships
30 
31### 2. Performance Metrics
32 
33- **Execution Time**: How long operations take
34- **Memory Usage**: Peak and average memory consumption
35- **CPU Utilization**: Processor usage patterns
36- **I/O Wait**: Time spent on I/O operations
37 
38### 3. Optimization Strategies
39 
40- **Algorithmic**: Better algorithms and data structures
41- **Implementation**: More efficient code patterns
42- **Parallelization**: Multi-threading/processing
43- **Caching**: Avoid redundant computation
44- **Native Extensions**: C/Rust for critical paths
45 
46## Quick Start
47 
48### Basic Timing
49 
50```python
51import time
52 
53def measure_time():
54    """Simple timing measurement."""
55    start = time.time()
56 
57    # Your code here
58    result = sum(range(1000000))
59 
60    elapsed = time.time() - start
61    print(f"Execution time: {elapsed:.4f} seconds")
62    return result
63 
64# Better: use timeit for accurate measurements
65import timeit
66 
67execution_time = timeit.timeit(
68    "sum(range(1000000))",
69    number=100
70)
71print(f"Average time: {execution_time/100:.6f} seconds")
72```
73 
74## Detailed patterns and worked examples
75 
76Detailed pattern documentation lives in `references/details.md`. Read that file when the navigation tier above is insufficient.
77 
78## Best Practices
79 
801. **Profile before optimizing** - Measure to find real bottlenecks
812. **Focus on hot paths** - Optimize code that runs most frequently
823. **Use appropriate data structures** - Dict for lookups, set for membership
834. **Avoid premature optimization** - Clarity first, then optimize
845. **Use built-in functions** - They're implemented in C
856. **Cache expensive computations** - Use lru_cache
867. **Batch I/O operations** - Reduce system calls
878. **Use generators** for large datasets
889. **Consider NumPy** for numerical operations
8910. **Profile production code** - Use py-spy for live systems
90 
91## Common Pitfalls
92 
93- Optimizing without profiling
94- Using global variables unnecessarily
95- Not using appropriate data structures
96- Creating unnecessary copies of data
97- Not using connection pooling for databases
98- Ignoring algorithmic complexity
99- Over-optimizing rare code paths
100- Not considering memory usage
101