Is Python's future currently at risk? (TLDR: No)


Calvin Spealman recently posted his thoughts on various aspects of where he sees computing in general heading, and his concerns about where Python may fit in that future.

I think his concerns are somewhat valid as far as specific market segments go, but I think they're overstating the case when it comes to "the future of Python", because I think his article takes a very narrow view of the computing field.

Smartphones and tablets are the new desktop (although the desktop won't go away, it will become limited to power users with demands for precision control and complex workflows). Python has long been relatively weak on the desktop when it comes to redistributing applications, due to the need to get the interpreter installed before it can be used. Microsoft's redistribution restrictions on their C runtime has made this all the more difficult when it comes to Windows.

We also made a fairly major misstep when we failed to appropriately advertise the addition of directory and zipfile execution support in Python 2.6 (bundle your code with all its dependencies except Python into a directory or zipfile and add a __main__.py file and the Python interpreter will execute it as if it was a script. With a zip file, you can even add a shebang line to the front and flag it it as executable and a POSIX shell will pass it to Python automatically if you run it directly. I haven't tried it, but the py launcher shipped with 3.3 should also handle such files). While we later went back and added the appropriate notice to the What's New in Python 2.6 documentation, and updated the command line guide in the documentation, this capability still isn't widely known.

The complaints about dynamic language overhead on mobile devices don't hold much water for me. Smartphones now are more powerful than desktops were less than a decade ago, and Mozilla's Boot2Gecko project holds a lot of potential. While battery technology doesn't advance as fast as computing technology, Moore's Law is leveraged in the mobile space to allow more to be done with less power, reproducing the desktop (and server!) trajectories where dynamic languages were initially derided as too slow, until the hardware caught up to get them to the point of being "fast enough".

However, Python's real strengths have long been server side technology, software development by non-programmers and as an embedded scripting engine for trusted plugins (rather than those that need to be strictly isolated from, for example, a core game engine or the host OS). And in those areas, it's still powering ahead.

Widespread adoption requires being taken for granted

Install a Linux distro. Which dynamic language interpreters are pre-installed? If you're using Debian or Fedora, it will be Python and Perl. The presence of those two can pretty much be taken for granted. Ruby probably won't be there, and a standalone Javascript interpreter certainly won't be.

Apple have expressed their support for Python by building tools that rely on it (with, as far as I know, Python being the only dynamic language interpreter shipped as part of Mac OS X. Update: I'm told Apple ship Perl and Ruby as well), and Microsoft ship their Python Tools for Visual Studio bundle. Google, of course, famously chose Python as the only dynamic language supported on their App Engine platform (and they currently employ Guido van Rossum and a number of other Python core developers).

gcc and gdb both let you write plugins, and your language choices are C/C++ or Python (plus Lisp in the gcc case). Many other infrastructure level tools are going the same way. Fedora's infrastructure is almost entirely written in Python, as is OpenStack.

If you're into multimedia development, Python will be a core part of your toolset, and Python is the key open source competitor to proprietary toolsets in the scientific community. The Natural Language Toolkit is a hugely powerful resource for many data mining applications, and Python is entwined deeply into the core of the financial sector as well.

Also, just as many years ago a lot of formal education program switched from C and C++ (or Pascal or Ada, etc) to Java for introductory programming courses, many are now switching to Python, pushing Java into the role of an enterprise language used only for large and complex applications where the development overhead can be justified to some degree. Businesses are getting to the point where they can choose Python as part of their technology base while being assured of a future pool of recruits that already know the language.

Informal education programs are also favouring Python as the first "real world" application language that people are introduced to. OLPC chose Python, as did RaspberryPI. Readability counts.

The Python Africa Tour has attracted quite a bit of interest, and I believe Africa plays host to its first PyCon later this year (in South Africa). Every other continent has now hosted multiple PyCon's each year in different countries and regions.

Only one kind of client

Things are substantially more competitive on the web service front, with Rails and Django going head-to-head, and Node.js attempting to play the "you can use the same language on the frontend and the backend!" card.

As far as Node.js goes, I'm firmly convinced that if Node.js was going to be a hugely popular server side framework, Twisted would have taken over the world by now. Callback based programming is just plain hard for most humans to wrap their heads around (often even harder than threaded programming) - hence the popularity of greenlets and gevent in the Python world, which permit the use of asynchronous IO capabilities with a threading-like programming style. The ongoing efforts around tweaking generator syntax and capabilities in Python core development could legitimately be summarised as "make it possible to write Twisted code in a way that doesn't hurt people's brains quite so much and without relying on the magical stack-switching assembly code needed for greenlets".

In this space, Python's strength really lies in its ability to step away from traditional web technologies. Want to talk over a serial port to a piece of lab equipment or radio modem? Sure, we can do that. Want to talk to telco gear through a custom C extension? Sure, we have a wide range of tools to support that, too, along with some great Asterisk bindings. Python also has many web framework options, like Pyramid and Flask, that let you be more easily be selective in your choice of components than Django does.

This is important, because I just spent the past weekend here at PyCon India. While smartphones are popular amongst the largely urban professionals that make up the web development community and those they regularly associate with, they're still only available to a vanishingly small percentage of the global population. Much of the rest of the world doesn't even have access to a desktop computer let alone a smartphone. What they do have though are ordinary mobile phones (aka cellphones, for any Americans in the audience).

Added to that is the fact that the majority of the world's population is illiterate - they can understand spoken instructions, and are sufficiently numerate to press numbers on a keypad to operate an Interactive Voice Response System, but they won't be operating a smartphone any time soon, even if one was available to them.

The interfaces and language capabilities you need to reach *that* audience look nothing like those you can use to reach the smartphone toting crowd.

And that's before we even get into the potential long term implications of verbal and tactile interfaces like Siri and Baxter.

No reason to relax

All that said, while Python's future is looking very, very bright from where I sit, that's no reason to relax and assume that future is assured. Python is far from perfect, and the same can be said for the ecosystem around us.

Jacob's Sunday keynote at PyCon India spoke about the need for Python's web community to work on embracing the real time web, and lowering the barriers to entry to providing network-based realtime interactivity in Python-based web applications. It's likely any such efforts will require an update to the WSGI standards to support a streaming IO component, in addition to the current request/response model.

Tools like Kivy, that aim to make it easier to write mobile applications in Python are also an important part of extending Python's reach into areas where it is currently weak.

The recent 3.3 release included several elements aimed at making things easier for beginners (especially those on Windows), including improved error messages, an option to modify PATH in the Windows installer and the Python launcher, while the entire Python 3 series is aimed at embracing Unicode as part of the core of the language, allowing it to better reach beyond its original audience of users whose native alphabets could be expressed within the constraints of ASCII or an 8-bit encoding.

3.3 also took some of the first steps in improving the "out of the box" packaging dependency management experience, by integrating virtual environment support and namespace packages (along with making empty __init__.py files optional).

Concurrency is a problem where the overall Python ecosystem has many more options than those provided by the CPython interpreter implementation. We do offer plenty of interesting tools, especially for embarrassingly parallel problems that fit nicely into the concurrent.futures execution model. The GIL does cause problems for particular workloads, and switching to Jython or IronPython to take advantage of the free-threaded JVM and CLR implementations isn't always going to be an option. I've written far more extensively on that topic, though, so I won't repeat that here.

We should also look at ways of making it easier for other languages to interoperate with Python without an intervening C interface. Perhaps Python should ship a pycall script like this one, that makes it easy to invoke Python functions directly in a pipeline or from another application (passing JSON data in via stdin, and receiving JSON data back via stdout). Conversely, better shell integration is always worth exploring.

And, of course, our journey in rebuilding the Unicode infrastructure is ongoing. Python 3.4 is likely to bring improvements in the ability to switch the encoding of a stream "mid-flight", as well as restoring some convenience APIs for the non-Unicode related uses of the encoding and decoding methods in Python 2.

So yes, there are plenty of areas where Python can, and should, and probably will, improve. But we shouldn't lose sight of the fact that many of the problems with Python (like binary distribution, dependency management and concurrency) are problems with software development generally, so there's nowhere for people to go that will magically make those issues disappear (or else they come at the price of losing out on many of Python's other advantages, or committing to a particular platform, or some other downside).

We're a conservative community by nature - we generally don't like blazing trails when it comes to language design. Instead, we're happy to let others rush ahead, letting them figure out where the pitfalls are, while we see what we can learn from their experience and integrate into Python's syntax, standard libraries, or the Python Package Index.