What’s New In Python 3.4

Author:R. David Murray <rdmurray@bitdance.com> (Editor)

This article explains the new features in Python 3.4, compared to 3.3. Python 3.4 was released on March 16, 2014. For full details, see the changelog.

See also

PEP 429 – Python 3.4 Release Schedule

Summary – Release Highlights

New syntax features:

  • No new syntax features were added in Python 3.4.

Other new features:

New library modules:

Significantly improved library modules:

Security improvements:

CPython implementation improvements:

Please read on for a comprehensive list of user-facing changes, including many other smaller improvements, CPython optimizations, deprecations, and potential porting issues.

New Features

PEP 453: Explicit Bootstrapping of PIP in Python Installations

Bootstrapping pip By Default

The new ensurepip module (defined in PEP 453) provides a standard cross-platform mechanism to bootstrap the pip installer into Python installations and virtual environments. The version of pip included with Python 3.4.0 is pip 1.5.4, and future 3.4.x maintenance releases will update the bundled version to the latest version of pip that is available at the time of creating the release candidate.

By default, the commands pipX and pipX.Y will be installed on all platforms (where X.Y stands for the version of the Python installation), along with the pip Python package and its dependencies. On Windows and in virtual environments on all platforms, the unversioned pip command will also be installed. On other platforms, the system wide unversioned pip command typically refers to the separately installed Python 2 version.

The pyvenv command line utility and the venv module make use of the ensurepip module to make pip readily available in virtual environments. When using the command line utility, pip is installed by default, while when using the venv module API installation of pip must be requested explicitly.

For CPython source builds on POSIX systems, the make install and make altinstall commands bootstrap pip by default. This behaviour can be controlled through configure options, and overridden through Makefile options.

On Windows and Mac OS X, the CPython installers now default to installing pip along with CPython itself (users may opt out of installing it during the installation process). Window users will need to opt in to the automatic PATH modifications to have pip available from the command line by default, otherwise it can still be accessed through the Python launcher for Windows as py -m pip.

As discussed in the PEP, platform packagers may choose not to install these commands by default, as long as, when invoked, they provide clear and simple directions on how to install them on that platform (usually using the system package manager).

Note

To avoid conflicts between parallel Python 2 and Python 3 installations, only the versioned pip3 and pip3.4 commands are bootstrapped by default when ensurepip is invoked directly - the --default-pip option is needed to also request the unversioned pip command. pyvenv and the Windows installer ensure that the unqualified pip command is made available in those environments, and pip can always be invoked via the -m switch rather than directly to avoid ambiguity on systems with multiple Python installations.

Documentation Changes

As part of this change, the Installing Python Modules and Distributing Python Modules sections of the documentation have been completely redesigned as short getting started and FAQ documents. Most packaging documentation has now been moved out to the Python Packaging Authority maintained Python Packaging User Guide and the documentation of the individual projects.

However, as this migration is currently still incomplete, the legacy versions of those guides remaining available as Installing Python Modules (Legacy version) and Distributing Python Modules (Legacy version).

See also

PEP 453 – Explicit bootstrapping of pip in Python installations
PEP written by Donald Stufft and Nick Coghlan, implemented by Donald Stufft, Nick Coghlan, Martin von Löwis and Ned Deily.

PEP 446: Newly Created File Descriptors Are Non-Inheritable

PEP 446 makes newly created file descriptors non-inheritable. In general, this is the behavior an application will want: when launching a new process, having currently open files also open in the new process can lead to all sorts of hard to find bugs, and potentially to security issues.

However, there are occasions when inheritance is desired. To support these cases, the following new functions and methods are available:

See also

PEP 446 – Make newly created file descriptors non-inheritable
PEP written and implemented by Victor Stinner.

Improvements to Codec Handling

Since it was first introduced, the codecs module has always been intended to operate as a type-neutral dynamic encoding and decoding system. However, its close coupling with the Python text model, especially the type restricted convenience methods on the builtin str, bytes and bytearray types, has historically obscured that fact.

As a key step in clarifying the situation, the codecs.encode() and codecs.decode() convenience functions are now properly documented in Python 2.7, 3.3 and 3.4. These functions have existed in the codecs module (and have been covered by the regression test suite) since Python 2.4, but were previously only discoverable through runtime introspection.

Unlike the convenience methods on str, bytes and bytearray, the codecs convenience functions support arbitrary codecs in both Python 2 and Python 3, rather than being limited to Unicode text encodings (in Python 3) or basestring <-> basestring conversions (in Python 2).

In Python 3.4, the interpreter is able to identify the known non-text encodings provided in the standard library and direct users towards these general purpose convenience functions when appropriate:

>>> b"abcdef".decode("hex")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
LookupError: 'hex' is not a text encoding; use codecs.decode() to handle arbitrary codecs

>>> "hello".encode("rot13")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
LookupError: 'rot13' is not a text encoding; use codecs.encode() to handle arbitrary codecs

>>> open("foo.txt", encoding="hex")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
LookupError: 'hex' is not a text encoding; use codecs.open() to handle arbitrary codecs

In a related change, whenever it is feasible without breaking backwards compatibility, exceptions raised during encoding and decoding operations are wrapped in a chained exception of the same type that mentions the name of the codec responsible for producing the error:

>>> import codecs

>>> codecs.decode(b"abcdefgh", "hex")
Traceback (most recent call last):
  File "/usr/lib/python3.4/encodings/hex_codec.py", line 20, in hex_decode
    return (binascii.a2b_hex(input), len(input))
binascii.Error: Non-hexadecimal digit found

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
binascii.Error: decoding with 'hex' codec failed (Error: Non-hexadecimal digit found)

>>> codecs.encode("hello", "bz2")
Traceback (most recent call last):
  File "/usr/lib/python3.4/encodings/bz2_codec.py", line 17, in bz2_encode
    return (bz2.compress(input), len(input))
  File "/usr/lib/python3.4/bz2.py", line 498, in compress
    return comp.compress(data) + comp.flush()
TypeError: 'str' does not support the buffer interface

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: encoding with 'bz2' codec failed (TypeError: 'str' does not support the buffer interface)

Finally, as the examples above show, these improvements have permitted the restoration of the convenience aliases for the non-Unicode codecs that were themselves restored in Python 3.2. This means that encoding binary data to and from its hexadecimal representation (for example) can now be written as:

>>> from codecs import encode, decode
>>> encode(b"hello", "hex")
b'68656c6c6f'
>>> decode(b"68656c6c6f", "hex")
b'hello'

The binary and text transforms provided in the standard library are detailed in Binary Transforms and Text Transforms.

(Contributed by Nick Coghlan in issue 7475, issue 17827, issue 17828 and issue 19619.)

PEP 451: A ModuleSpec Type for the Import System

PEP 451 provides an encapsulation of the information about a module that the import machinery will use to load it (that is, a module specification). This helps simplify both the import implementation and several import-related APIs. The change is also a stepping stone for several future import-related improvements.

The public-facing changes from the PEP are entirely backward-compatible. Furthermore, they should be transparent to everyone but importer authors. Key finder and loader methods have been deprecated, but they will continue working. New importers should use the new methods described in the PEP. Existing importers should be updated to implement the new methods. See the Deprecated section for a list of methods that should be replaced and their replacements.

Other Language Changes

Some smaller changes made to the core Python language are:

  • Unicode database updated to UCD version 6.3.
  • min() and max() now accept a default keyword-only argument that can be used to specify the value they return if the iterable they are evaluating has no elements. (Contributed by Julian Berman in issue 18111.)
  • Module objects are now weakref‘able.
  • Module __file__ attributes (and related values) should now always contain absolute paths by default, with the sole exception of __main__.__file__ when a script has been executed directly using a relative path. (Contributed by Brett Cannon in issue 18416.)
  • All the UTF-* codecs (except UTF-7) now reject surrogates during both encoding and decoding unless the surrogatepass error handler is used, with the exception of the UTF-16 decoder (which accepts valid surrogate pairs) and the UTF-16 encoder (which produces them while encoding non-BMP characters). (Contributed by Victor Stinner, Kang-Hao (Kenny) Lu and Serhiy Storchaka in issue 12892.)
  • New German EBCDIC codec cp273. (Contributed by Michael Bierenfeld and Andrew Kuchling in issue 1097797.)
  • New Ukrainian codec cp1125. (Contributed by Serhiy Storchaka in issue 19668.)
  • bytes.join() and bytearray.join() now accept arbitrary buffer objects as arguments. (Contributed by Antoine Pitrou in issue 15958.)
  • The int constructor now accepts any object that has an __index__ method for its base argument. (Contributed by Mark Dickinson in issue 16772.)
  • Frame objects now have a clear() method that clears all references to local variables from the frame. (Contributed by Antoine Pitrou in issue 17934.)
  • memoryview is now registered as a Sequence, and supports the reversed() builtin. (Contributed by Nick Coghlan and Claudiu Popa in issue 18690 and issue 19078.)
  • Signatures reported by help() have been modified and improved in several cases as a result of the introduction of Argument Clinic and other changes to the inspect and pydoc modules.
  • __length_hint__() is now part of the formal language specification (see PEP 424). (Contributed by Armin Ronacher in issue 16148.)

New Modules

asyncio

The new asyncio module (defined in PEP 3156) provides a standard pluggable event loop model for Python, providing solid asynchronous IO support in the standard library, and making it easier for other event loop implementations to interoperate with the standard library and each other.

For Python 3.4, this module is considered a provisional API.

See also

PEP 3156 – Asynchronous IO Support Rebooted: the “asyncio” Module
PEP written and implementation led by Guido van Rossum.

ensurepip

The new ensurepip module is the primary infrastructure for the PEP 453 implementation. In the normal course of events end users will not need to interact with this module, but it can be used to manually bootstrap pip if the automated bootstrapping into an installation or virtual environment was declined.

ensurepip includes a bundled copy of pip, up-to-date as of the first release candidate of the release of CPython with which it ships (this applies to both maintenance releases and feature releases). ensurepip does not access the internet. If the installation has Internet access, after ensurepip is run the bundled pip can be used to upgrade pip to a more recent release than the bundled one. (Note that such an upgraded version of pip is considered to be a separately installed package and will not be removed if Python is uninstalled.)

The module is named ensurepip because if called when pip is already installed, it does nothing. It also has an --upgrade option that will cause it to install the bundled copy of pip if the existing installed version of pip is older than the bundled copy.

enum

The new enum module (defined in PEP 435) provides a standard implementation of enumeration types, allowing other modules (such as socket) to provide more informative error messages and better debugging support by replacing opaque integer constants with backwards compatible enumeration values.

See also

PEP 435 – Adding an Enum type to the Python standard library
PEP written by Barry Warsaw, Eli Bendersky and Ethan Furman, implemented by Ethan Furman.

pathlib

The new pathlib module offers classes representing filesystem paths with semantics appropriate for different operating systems. Path classes are divided between pure paths, which provide purely computational operations without I/O, and concrete paths, which inherit from pure paths but also provide I/O operations.

For Python 3.4, this module is considered a provisional API.

See also

PEP 428 – The pathlib module – object-oriented filesystem paths
PEP written and implemented by Antoine Pitrou.

selectors

The new selectors module (created as part of implementing PEP 3156) allows high-level and efficient I/O multiplexing, built upon the select module primitives.

statistics

The new statistics module (defined in PEP 450) offers some core statistics functionality directly in the standard library. This module supports calculation of the mean, median, mode, variance and standard deviation of a data series.

See also

PEP 450 – Adding A Statistics Module To The Standard Library
PEP written and implemented by Steven D’Aprano

tracemalloc

The new tracemalloc module (defined in PEP 454) is a debug tool to trace memory blocks allocated by Python. It provides the following information:

  • Trace where an object was allocated
  • Statistics on allocated memory blocks per filename and per line number: total size, number and average size of allocated memory blocks
  • Compute the differences between two snapshots to detect memory leaks

See also

PEP 454 – Add