Major Python changes

Note: Please see the Recent Python Changes appendix in the 2nd edition of Programming Python (published March 2001), for a much more comprehensive and up to date list of Python language and library changes. The list below is no longer maintained.

This page lists and briefly describes major changes in Python between the time the first edition of Programming Python was published (10/96, Python 1.3), and the last update to this page (2/99, Python 1.5.2).

Most of these changes are minor, and by computer book standards this book is still fairly up to date (especially since it's more about how to really use Python, than about memorizing language details). In fact, most recent Python changes are optional additions which you can safely ignore when just starting out. Make sure you understand the core ideas presented in the book, before tackling the new additions.

On the other hand, Python is a growing system, and some of its recent changes impact sections of the book. I've grouped Python changes below into categories. Note: this list is a work in progress, and isn't necessarily complete (and may be out of date as soon as Guido releases another version), so be sure to also see the "What's New" release notes if you install a newer version of Python (or click here for more details on changes in Python 1.5).

Also note that there are a number of new documentation sources available, including two new O'Reilly books (Learning Python, and the Python Pocket Reference), as well as updated library, language, and C API documentation at Python's web site. The pocket reference book is partly intended to provide an up-to-date version of the "Mini-Reference" appendix in Programming Python.

Related pages:

Index for this page (XXX - more to come):


Core Language Changes

A handful of new features, most of which are optional, backward-compatible with earlier releases, and designed by and for Python wizards. I recommend that beginners stick to the true core language described in the book when first starting out, and postpone most of the new features listed here until they're comfortable with the basics.



Pseudo-private class attributes

Python now provides a name-mangling protocol which provides a limited sort of "private" mechanism for hiding class names (attributes). A name of the form "__X" inside a class statement is automatically changed by Python to "_Class__X", where "Class" is the name of the class being defined by the statement.

The upshot is that by prefixing such attributes with the name of the enclosing class, they are localized to that class, and will be distinct from similar names used by other classes. This isn't true "privacy" in the C++ sense--it avoids some name clashes, but class clients can still get to "__X" names if they know the class name. It's also an entirely optional feature, and a compromise for those wanting C++-style access restrictions.

With this extension, there are now 3 access rules in Python:



Class exceptions

Exceptions may now take the form of class (and class instance) objects. The intent is to support exception categories: since an except clause will match a raised exception if it names the raised class or any of its superclasses, specifying superclasses allows try statements to catch broad categories without listing all members explicitly.

Moreover, Python's standard (built-in) exceptions are now classes (instead of strings as before), and have been organized into a shallow class hierarchy:

For the most part, this change has no impact on existing programs, unless they try to concatenate standard exceptions (by assuming they're still strings). To get the old string exceptions, use the "-X" flag on the Python command line; alternatively, change problematic programs to wrap exceptions in a str() call before concatenating (e.g., str(X) + str(Y)).


Package imports

Due to demand, Guido implemented a new, standard way to import packages of modules, which replaces the older (experimental) "ni" package support module. Roughly, packages are directories of Python module files and other packages, and package imports are strings of dotted names (qualifications) which reflect the directory structure on your machine.

For instance, "import package.package.module" loads a module nested 2-levels deep in packages (directories), and "from package.package import module" has a similar effect. Unlike the older "ni", the new package support is always available (without running special imports), and requires each package directory to have a "__init__.py" module file. See www.python.org for more details.


New "assert" statement

Python 1.5 added a new statement:

assert test [, value]
is the same as:

if __debug__:
    if not test:
        raise AssertionError, value

but is easier on the eyes and keyboard. Assertions are mostly meant for debugging, but can also be used to specify program constraints.


Reserved word changes

"assert" was added to the list of Python reserved words; "access" was taken away (it's now been deprecated in earnest).


New dictionary methods

A few convenience methods were added to the built-in dictionary object to avoid the need for manual "for" loops: D.clear(), D.copy(), D.update(), and D.get(). The first two empty and copy dictionaries. D1.update(D2) is equivalent to the for loop "for k in D2.keys(): D1[k] = D2[k]". D.get(k) returns D[k] if it exists and None (or the optional second argument) if not.


"Raw" string constants

In support of regular expressions, Python now allows string constants to be written in the form r"...\...", which works like a normal string, except that Python leaves any backslashes (\) in the quotes alone; they remain as literal "\" characters in the string, rather than being interpreted as special escape codes by Python.


Complex number type

Python now supports complex number constants (e.g., 1+3j), and complex arithmetic operations (normal math operators, plus a "cmath" module with many of the "math" module's functions for complex numbers).


Printing cyclic objects doesn't core dump

Objects created with code like L.append(L) are now detected and printed specially by the interpreter. In the past, trying to print cyclic objects caused the interpreter to loop (recursively), which led to a core dump if you didn't kill it fast enough.


"raise" without arguments: re-raise

A raise statement without any exception or extra-data arguments is now syntactically legal, and makes Python re-raise the most recently raised uncaught exception.


"raise" forms for class exceptions

Because exceptions can now be either string objects, or classes and class instances, you can now use any of the following raise statement forms (the last 3 of which are for backward compatibility with earlier releases where all built-in exceptions were strings):

raise string           # matches except with same string object
raise string, data     # same, with optional data

raise class, instance  # matches except with class or its superclass
raise instance         # same as: raise instance.__class__, instance

raise                  # reraise last exception

raise class            # same as: raise class(), and: raise class, instance
raise class, arg       # same as: raise class(arg)
raise class, (arg,...) # same as: raise class(args...)



Power operator "X ** Y"

The new "**" binary operator computes the left operand raised to the power of the right operand. It works much like the built-in "pow" function.


Generalized sequence assignments

In an assignment ("=" statement, and other assignment contexts), you can now assign any sort of sequence on the right to a list or tuple on the left (e.g., "(A,B) = seq", "[A,B] = seq"). In the past, the sequence types had to match.


It's faster

Guido & Co. worked on optimizing the virtual machine, especially in Python 1.5. How much faster depends on what sort of test you run, of course, but Python 1.5 has been clocked at almost twice as fast as its predecessors on the Lib/test/pystone.py benchmark.



Library Changes

There have been lots of changes and additions to the libraries in the last two years. The list here is incomplete, and mostly focused on changes which impact examples and discussion in the book. Since the book isn't an exhaustive treatment of the libraries (there are hundreds of library modules), be sure to see the documentation for library modules you use if they're not described here.



"dir(X)" now works on more objects

The built-in dir function now reports attributes for modules, classes, class instances, as well as built-in objects like lists, dictionaries, and files. You don't need to use members like __methods__ (but still can).


New conversions: "int(X)", float(X)", list(S)"

The "int" and "float" built-in functions now accept string arguments, and convert from string to numbers exactly like "string.atoi/atof". The new "list(S)" built-in function converts any sequence to a list, much like the older "map(S, None)" trick.


New "re" regular expression module

A new regular expression module, "re", offers full-blown Perl-style regular expression matching. The older "regex" module described in the book is still available, and provides a simpler regular expression matching tool. But the new "re" module provides a more powerful matching system; for instance, it supports "non-greedy" matches, and most of Perl's special regular expression syntax, including a variety of character classes. See the "re" module description at the www.python.org doc page, or the Python Pocket Reference book.


spiltfields/joinfields -> split/join

The spilt and join functions in the string module were generalized to do the same work as the original splitfields and joinfields. In other words, you don't need to type the "fields" bit any more--just pass in delimiter arguments to split and join.


Persistence: unpickler no longer calls __init__

Beginning in Python 1.5, the pickle module's unpickler (loader) no longer calls class __init__ methods to recreate pickled class instance objects. This means that classes no longer need to have defaults for all constructor arguments to be used for persistent objects. To force Python to call the __init__ method (as it did before), classes must provide a __getinitargs__ method; see the library manual for details.


Pickle coded in C: cpickle

An implementation of the pickle module in C is now a standard part of Python. It's called "cpickle", and is reportedly many times faster than the original pickle module.


anydbm.open now expects a 'c' second argument

To create a new or open an existing file in read+write mode, pass a 'c' in argument 2 to anydbm.open. This changed as of Python 1.5.2; passing a 'c' now does what passing no second argument used to do (the second argument now defaults to 'r'). This does not impact shelve.open.


rand module replaced by random module

The title says it all: "rand" is now deprecated; use "random" instead.


Assorted Tkinter changes

Besides becoming portable and getting native look-and-feel on all major platforms, there has been a variety of growth in the Tkinter GUI interface (Python's Tk wrapper):

CGI module interface change

The CGI interface used in the Appendix A CGI mailto example has become "deprecated" (which means you can still use it, but the newer interface is recommended). In basic terms, the older interface looks like this:

The new interface is: See the cgi module's documentation for more details.


"site.py", "user.py", and PYTHONHOME

Scripts automatically run by Python on start-up, used to tailor the initial configuration. (XXX - more to come.)


Assignment to "os.environ[key]" calls putenv

Assigning to a key in the os.environ dictionary really updates the corresponding environment variable in the C environment now; it triggers a call to the C library's "putenv" routine, such that the changes are reflected in integrated C code layers, as well as the environment of any child processes spawned by the Python program. Moreover, the "putenv" routine is now exposed in the os module ("os.putenv"); be sure to consider the extension module example in chapter 2 for illustration purposes only.


New sys.exc_info() tuple

The new exc_info() function in the sys module returns a tuple that has values corresponding to the older sys.exc_type and sys.exc_value, shown in the book. The difference is that the older names represent a single global exception, and exc_info() is specific to the calling thread.


The new "operator" module

There is a new standard module called "operator", which provides functions which implement most of the built-in Python expression operators. For instance, "operator.add(X,Y)" does the same thing as "X+Y", but because operator module exports are functions, they are sometimes handy to use in things like "map", without having to create a function or use a "lambda" form.



Tool Changes

This section is a sampling of changes in Python tools and Python-related systems, as an extension to Appendix A. This is a partial and necessarily subjective list of exciting developments on the Python application fronts, and I'm positive I've missed a few; be sure to see www.python.org for others.



JPython: A Python-to-Java compiler

A new system worth noting: the "JPython" package is an alternative Python implementation, which compiles Python programs to Java Virtual Machine (JVM) bytecode, and provides hooks for integrating Python and Java programs. JPython makes Python an excellent scripting tool for Java-based Web systems. Like any radical development, it represents good news and/or bad news, depending on your point of view; in the interest of fairness, I'd like to summarize both sides here.

The good news: Because JPython compiles Python programs to Java byte codes, they can be run on any system with an installed JVM. JPython also provides additional tools for accessing Java objects and packages, including AWT. For instance, JPython allows Python programs to implement client-side applets, which may create portable AWT and Swing GUIs in standard Net browsers. JPython was also recently certified as "100% Pure Java".

So where does Python/Java integration fit in with Python's world view? In general, that's really the same as asking about Python/C++ integration. Now that Java has been around for a few years, it's generally classified as a systems language (in the same category as C and C++), not as a scripting language (which is Python's domain). Java is simply too complex to be useful for scripting work.

But as the book stresses, you often need both kinds of tools--a systems language for performance, and a scripting tool for usability. By integrating a true scripting language like Python with Java, programmers can realize all the same scripting benefits the book reports for integrating Python with C or C++.

JPython becomes more interesting when applied to web-based systems. For instance, by compiling Python to JVM code, it can be run as client-side code in any Java-aware browser, and have access to Java libraries. JPython is probably less useful outside of web-based systems (if Java has all the complexity of C++ and all the performance problems of an interpreted language, why not just use C++ for compute-intensive components?). But for systems that must run on the web, JPython provides an interesting and portable scripting option.

For a more complete and compelling argument for JPython than I can make here, read Guido's overview article about JPython, in developer.com (8/98).

The bad news: On the downside, JPython has a limited scope, isn't yet compatible with the standard C implementation of Python, and is somewhat slower due to the extra interpretation levels. For instance, JPython:

Because of its current compatibility problems and limited scope (it's not applicable outside JVM environments), JPython is probably best seen as a useful tool for scripting Java based systems which run on the web, and an alternative Python implementation.

I'm being careful to present both its pros and cons here, mostly because there have been some attempts to position JPython as a replacement for standard Python--an arguably odd proposal which would alienate tens of thousands of current Python users! But despite its tradeoffs, JPython is an amazing tool for Java programmers, and merits the attention of anyone involved with Java-based systems. For more details, see the JPython page.


MS-Windows ports: COM, Tkinter

The COM interfaces in the PythonWin ports have evolved substantially since the descriptions in Appendix A (where they were called OLE interfaces). PythonWin and COM make Python an excellent alternative to Visual Basic for scripting Windows applications. See www.python.org for details. Fredrik Lundh also now makes available Python executables for Windows (9x/NT), with things like the Tkinter GUI interface for Windows and cPickle preinstalled. See www.pythonware.com for links.


SWIG growth, C++ shadow classes

The SWIG system briefly described in chapters 2 and 15 has grown to become a primary extension writers tool. It also incorporated something called "shadow classes", which are equivalent to what the book describes as "wrapper" or "stub" classes in chapter 15--Python classes which wrap C++ interface extension modules or types, such that C++ classes look like Python classes to Python programs. SWIG can automatically generate C++ wrappers (both a C++ interface and a Python wrapper class), given C++ type signature information.


Bobo: Python objects for the Web

A system for publishing Python objects on the Web has grown to become a popular tool for CGI script writers and Web hackers. See http://www.digicool.com/releases/bobo for details.


HTMLGen: making HTML from Python classes

A tool for generating correct HTML files (for Web page layout) from Python class object trees has grown to maturity. See www.python.org for more details.


Pmw: Python mega-widgets for Tkinter

The Pmw system provides powerful, higher-level widgets for Tkinter-based GUIs in Python; it's worth checking out if you do any non-trivial Tkinter work. See python.org for details.


IDLE: a Python Integrated Development Environment GUI

Python now ships with a new point-and-click development interface named IDLE. Written in Python using the Tkinter GUI library, IDLE either comes in the source library's Tools directory, or is automatically installed with Python itself (on Windows). IDLE offers a syntax-coloring text editor, a graphical debugger, an object browser, and much more. If you have a Python with Tk support enabled, and are used to more advanced development interfaces, IDLE provides a nice feature-rich alternative to the traditional Python command line. IDLE is still evolving as I write this; it does not yet include support for point-and-click GUI construction.


Other tool growth: PIL, NumPy, Database API

The PIL image processing and NumPy numeric programming systems have matured considerably since the book was written. And that portable database API for Python which was hinted at in the "Futurisms" appendix has actually come together. See www.python.org for details (and/or www.pythonware.com for PIL).



Python/C Integration API Changes

A variety of improvements and extensions, mostly aimed at simplifying the task of embedding Python in C programs, and supporting threads at the C API level.



A single "Python.h" header file

All useful Python symbols are now exported in the single "Python.h" header file; no other header files need be imported (even when embedding--the extra parser and import headers aren't needed).


A single "libpython*.a" C library file

All Python interpreter code is now packaged in a single library file when you make Python. For instance, under Python 1.5, you need only link in "libpython1.5.a" when embedding Python (instead of the older 4 libraries plus .o's scheme). See file "Demo/embed/Makefile" in the Python1.5 source distribution for an example.


The "Great (Grand) Renaming" is complete

All exposed Python symbols now start with a "Py" prefix. For instance, "eval_input"/"file_input" parser mode constants for embedding code strings have been renamed to Py_eval_input and Py_file_input. Similarly, the "getprogramname" function which optionally appears in embedding examples is now called Py_GetProgramName, though you can instead call Py_SetProgramName(name). When in doubt, grep for old symbols in the Include directory of the source distribution.


Threading support, multiple interpreters

A handful of new API tools provide better support for threads when embedding Python. For instance, there are tools for finalizing Python (Py_Finalize), and creating "multiple interpreters" (Py_NewInterpreter).

In many cases, multiple namespaces are sufficient to isolate names used in independent system components (and easier to manage than multiple interpreters and threads). But in some threaded programs, it's also useful to have one copy of system modules and structures per thread and this is where multiple interpreters come in handy (e.g., the "sys" module: without one copy per thread, imports might find an already loaded module in the sys.modules table if it was imported by a different thread). See the new C API documentation at the "doc" link at www.python.org for details.

Note that you still can call Py_Initialize more than once (despite the comments in pythonrun.c in 1.5), so the higher-level embedding API developed in chapter 15 still works. Py_Initialize checks a static flag to immediately return if it's already been called and Py_Finalize hasn't.


New Python C API documentation

Speaking of docs: there is a new reference manual which documents major C API tools and behavior. It's not complete yet, but is a useful start. See the "doc" link at www.python.org, or click here.



Back to the errata page
Back to my homepage



[Home page] Books Code Blog Python Author Train Find ©M.Lutz