TwistedElephant
===============

:author: bbum
:email: bbum@mac.com
:copyright: 2007 Bill Bumgarner, All Rights Reserved.
:license: MIT License (http://en.wikipedia.org/wiki/MIT_License)

.. admonition:: Abstract

    A simple HTTP based interface for introspecting GC state of a `Twisted`_ based process.  Through a point-and-click interface, TwistedElephant makes it easy to explore the current object graph realized within the Python VM.   One click to see all objects that refer to or are referred to by any particular object, including the ability to click on any of those objects to see their current connectivity.  Patches are certainly welcome.
    
.. warning::

    This is a complete and utter hack.  But it worked well enough for my purposes.   Specifically, I hacked on it until it could solve a particular memory management problem.   Development stoppped at that point.   There is an awful lot of easy stuff that could be done to make this better.

.. contents::

Installation
------------

TwistedElephant uses the standard python distutils packaging mechanism.

::

    sudo python setup.py install

Activation
----------

To use TwistedElephant, simply import the module and activate it.  Once activated, it will listen on the specified port for HTTP requests.   TwistedElephant is otherwise completely passive in its operation.

Testing & Usage
---------------

The `elephant/__init__.py` can be executed in a shell to cause TwistedElephant to fire up in about the simplest `Twisted`_ server possible.

::

    python elephant/__init__.py
    http://localhost:3535/

Connecting to the URL shown will present a simple [broken] UI via which you can poke around the current state of the Python garbage collector.   The initial display will typically show no objects as TwistedElephant automatically sets the MARK to exclude all objects that existed at the time of first load.  This prevents the first load of the web page from taking a long long time if there are already 10s of thousands of active objects.

Namely, you can:

[LIST] 

    Shows a list of all objects currently tracked by the collector.   Objects created prior to the last MARK are excluded.   Strings, dictionaries, and anything that cannot be weakref'd are also excluded.

[REFRESH]

    Refresh the display.
    
[MARK]

    Exclude all objects currently being tracked.

[CLEAR]

    Clear the list of excluded objects.   Note that 

[RAW]

    Dump a RAW list of **all objects in the runtime**.  Includes strings, dicts and, well, everything that the GC subsystem is tracking.   The raw list includes the id() of every object and can be used in conjunction with `di`_ to see referrers and referents. 

Object Graph Navigation
-----------------------

In any of the object list modes, you can click on the *Object ID* of an object to see the referrers to and referents of that object.   The URL for any particular object will look something like::

    http://localhost:3535/ObjectDetail/7173104

You can replace the *Object ID* at the end of the URL with the ID of any object found in the GC log produced by the RAW link.  If you have `di`_ installed, this enables the ability to follow referrers and referents for objects that cannot be weak referenced.  Without `di`_, many objects will produce a *this object no longer appears to exist* warning if they cannot be found in TwistedElephant's weak reference based tracking code.

.. caution::

    With `di`_ active, pasting in the object ID of an object that has been collected may very likely crash the Python VM or cause surprising behavior, such as returning a completely different object than the one expected.   `di`_ quite effectively uses object IDs for purposes Guido never intended.  Using `di`_ outside of the context of this kind of a debugging tool is exceedingly risky.   Don't do it.
    
Integrating Into an Existing Twisted Application
------------------------------------------------

TwistedElephant can be used with just about any `Twisted`_ based application.  Though it has a web based "interface", TwistedElephant can be used in non-HTTP servers without a problem.

::

    import TwistedElephant
    TwistedElephant.installElephant(port=6398)

TwistedElephant will print a localhost based URL upon successful activation.   Note that TwistedElephant uses server relative URLs throughout its implementation and, thus, you can substitute a remote server's hostname for localhost and all should work fine.


.. _di: http://www.friday.com/bbum/2007/08/24/python-di/
.. _Twisted: http://www.twistedmatrix.com/