python: line by line profiling (time, memory)

python: line by line profiling (time, memory)

Line-By-Line Profiling with %lprun

pip install line_profiler

In notebook

%load_ext line_profiler
%lprun -f slow_func slow_func(value)
Timer unit: 1e-09 s

Total time: 0.000134558 s
File: /tmp/ipykernel_70058/363435982.py
Function: slow_func at line 1

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
     1                                           def slow_func(numbers):
     2         1       1441.0   1441.0      1.1      filtered_numbers = []
     3       101      42213.0    418.0     31.4      for num in numbers:
     4       100      54900.0    549.0     40.8          if num % 2 != 0:
     5        50      33519.0    670.4     24.9              filtered_numbers.append(num)
     6         1       2025.0   2025.0      1.5      even_total = sum(filtered_numbers)
     7         1        460.0    460.0      0.3      return even_total

Profiling Memory with %mprun

pip install memory_profiler

Load extension in notebook

%load_ext memory_profiler
%mprun -f slow_func slow_func(value)
Filename: /home/pratapa/mprun_demo.py

Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
     1     63.2 MiB     63.2 MiB           1   def slow_func(numbers):
     2     63.2 MiB      0.0 MiB           1       filtered_numbers = []
     3     63.2 MiB      0.0 MiB         101       for num in numbers:
     4     63.2 MiB      0.0 MiB         100           if num % 2 != 0:
     5     63.2 MiB      0.0 MiB          50               filtered_numbers.append(num)
     6     63.2 MiB      0.0 MiB           1       even_total = sum(filtered_numbers)
     7     63.2 MiB      0.0 MiB           1       return even_total

def slow_func(numbers):
    filtered_numbers = []
    for num in numbers:
        if num % 2 != 0:
            filtered_numbers.append(num)
    even_total = sum(filtered_numbers)
    return even_total

Also see %prun and %memit


14. Apr 2024