Memory Profiling with Pyrasite and Heapy

2017-09-17 Discuss

1. Build or install Pyrasite from the develop branch in the GitHub repo. The PyPi package does not have the latest code, which allows you to control the IPC timeout.

   1. https://github.com/lmacken/pyrasite

2. Install profiling tools.

   sudo apt-get update
   sudo apt-get install pyrasite guppy
   

3. Enable ptrace, so that you can inject into the process.

   echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope
   

4. Connect to the running process.

   export PYRASITE_IPC_TIMEOUT=10  # default is 5 seconds
   
   /apps/python/bin/pyrasite-shell 3640
   Pyrasite Shell 2.0
   Connected to '...'
   Python 2.7.12 (default, Nov 19 2016, 06:48:10)
   [GCC 5.4.0 20160609] on linux2
   Type "help", "copyright", "credits" or "license" for more information.
   (DistantInteractiveConsole)
   
   >>>
   

5. Enumerate the threads in the current process.

   import sys, traceback
   
   for thread, frame in sys._current_frames().items():
       print('Thread 0x%x' % thread)
       traceback.print_stack(frame)
       print()
   

6. Profile process memory with heapy.

   from guppy import hpy
   hp = hpy()
   hp.heap()
   

7. Debug the pyrasite process, if needed.

   /apps/python/bin/pyrasite --verbose <pid> helloworld.py