contextlib2 0.4: Now with ExitStack!

Inspired by Michael Foord's efforts with unittest2, contextlib2 is a PyPI package where I am working on some new additions to the standard library's contextlib module for Python 3.3.

The most interesting of those is a replacement for the ill-fated contextlib.nested API.

If you use Python 3.2 today, you'll find that the contextlib.nested API doesn't even exist any more. The reason it was deprecated and removed is because it didn't play well with context managers that acquired their resources in __init__ rather than __enter__ (such as Python's own file objects and many other resources where using a with statement for management is an optional convenience rather than being mandatory).

The simplest example where the old API caused problems was opening a list of files and then using contextlib.nested to close them all when the operation was complete - if opening any later file threw an exception (e.g. due to a permissions error or a bad file name), then all of the earlier files would remain open until the garbage collector got around to cleaning them up. Not a huge problem on CPython with its refcounting based GC, but a far cry from the deterministic resource cleanup that context managers are supposed to offer.

Since the deprecation and removal of contextlib.nested, there have been assorted replacement proposals of varying levels of sophistication posted to the Python ideas mailing list. The new ExitStack API in this release is my own latest effort, and the first that I've liked well enough to seriously consider as a candidate for inclusion in the standard library module.

The idea behind providing the ExitStack API is for the standard library to focus specifically on handling the one particularly tricky part of dealing with context managers programmatically: unwinding the context stack correctly, ensuring that exceptions are processed exactly as if any context managers involved had been used in an appropriate series of nested with statements.

A couple of convenience methods are included (one that enters a context manager in addition to pushing the exit method on to the stack, as well as a simple callback registration method inspired by unittest.TestCase.addCleanup), but the features of the API are otherwise fairly minimal.

This low level dynamic API can then be used by developers to create their own higher level convenience APIs, as suggested in some of the examples in the documentation.

A few specific design notes:

• The name ExitStack came about because the object is literally a stack of exit method references (or callback wrappers that behave like exit methods). Earlier variants were ContextStack (too narrow, since you can use the stack for standalone callbacks) and CallbackStack (too broad, since the stored callbacks specifically have the signature of exit methods)
• The push() method accepts exit methods directly, since those are what actually gets stored on the stack. Ducktyping to also accept objects with an __exit__() method is convenient without being confusing (I hope).
• The enter_context() method uses the longer name because the shorter version is too easy to confuse with the stack's own __enter__() method.

If you have any questions about the ExitStack design, this is the place to ask. If you find any bugs or other defects, head over to the issue tracker.