What’s New In Python 3.4
|Author:||R. David Murray <firstname.lastname@example.org> (Editor)|
This article explains the new features in Python 3.4, compared to 3.3. Python 3.4 was released on March 16, 2014. For full details, see the changelog.
PEP 429 – Python 3.4 Release Schedule
Summary – Release Highlights
New syntax features:
- No new syntax features were added in Python 3.4.
Other new features:
- pip should always be available (PEP 453).
- Newly created file descriptors are non-inheritable (PEP 446).
- command line option for isolated mode (issue 16499).
- improvements in the handling of codecs that are not text encodings (multiple issues).
- A ModuleSpec Type for the Import System (PEP 451). (Affects importer authors.)
marshalformat has been made more compact and efficient (issue 16475).
New library modules:
asyncio: New provisional API for asynchronous IO (PEP 3156).
ensurepip: Bootstrapping the pip installer (PEP 453).
enum: Support for enumeration types (PEP 435).
pathlib: Object-oriented filesystem paths (PEP 428).
selectors: High-level and efficient I/O multiplexing, built upon the
selectmodule primitives (part of PEP 3156).
statistics: A basic numerically stable statistics library (PEP 450).
tracemalloc: Trace Python memory allocations (PEP 454).
Significantly improved library modules:
- Single-dispatch generic functions in
pickleprotocol 4 (PEP 3154).
multiprocessingnow has an option to avoid using os.fork on Unix (issue 8713).
contentmanager, and a new
EmailMessage) that simplify MIME handling (issue 18891).
pydocmodules are now capable of correct introspection of a much wider variety of callable objects, which improves the output of the Python
ipaddressmodule API has been declared stable
- Secure and interchangeable hash algorithm (PEP 456).
- Make newly created file descriptors non-inheritable (PEP 446) to avoid leaking file descriptors to child processes.
- New command line option for isolated mode, (issue 16499).
multiprocessingnow has an option to avoid using os.fork on Unix. spawn and forkserver are more secure because they avoid sharing data with child processes.
multiprocessingchild processes on Windows no longer inherit all of the parent’s inheritable handles, only the necessary ones.
- A new
hashlib.pbkdf2_hmac()function provides the PKCS#5 password-based key derivation function 2.
- TLSv1.1 and TLSv1.2 support for
- Retrieving certificates from the Windows system cert store support for
- Server-side SNI (Server Name Indication) support for
ssl.SSLContextclass has a lot of improvements.
- All modules in the standard library that support SSL now support server
certificate verification, including hostname matching
ssl.match_hostname()) and CRLs (Certificate Revocation lists, see
CPython implementation improvements:
- Safe object finalization (PEP 442).
- Leveraging PEP 442, in most cases module globals are no longer set to None during finalization (issue 18214).
- Configurable memory allocators (PEP 445).
- Argument Clinic (PEP 436).
Please read on for a comprehensive list of user-facing changes, including many other smaller improvements, CPython optimizations, deprecations, and potential porting issues.
PEP 453: Explicit Bootstrapping of PIP in Python Installations
Bootstrapping pip By Default
ensurepip module (defined in PEP 453) provides a standard
cross-platform mechanism to bootstrap the pip installer into Python
installations and virtual environments. The version of
with Python 3.4.0 is
pip 1.5.4, and future 3.4.x maintenance releases
will update the bundled version to the latest version of
pip that is
available at the time of creating the release candidate.
By default, the commands
pipX.Y will be installed on all
platforms (where X.Y stands for the version of the Python installation),
along with the
pip Python package and its dependencies. On Windows and
in virtual environments on all platforms, the unversioned
will also be installed. On other platforms, the system wide unversioned
pip command typically refers to the separately installed Python 2
The pyvenv command line utility and the
module make use of the
ensurepip module to make
available in virtual environments. When using the command line utility,
pip is installed by default, while when using the
API installation of
pip must be requested explicitly.
For CPython source builds on POSIX systems,
make install and
make altinstall commands bootstrap
default. This behaviour can be controlled through configure options, and
overridden through Makefile options.
On Windows and Mac OS X, the CPython installers now default to installing
pip along with CPython itself (users may opt out of installing it
during the installation process). Window users will need to opt in to the
PATH modifications to have
pip available from the command
line by default, otherwise it can still be accessed through the Python
launcher for Windows as
py -m pip.
As discussed in the PEP, platform packagers may choose not to install these commands by default, as long as, when invoked, they provide clear and simple directions on how to install them on that platform (usually using the system package manager).
To avoid conflicts between parallel Python 2 and Python 3 installations,
only the versioned
pip3.4 commands are bootstrapped by
ensurepip is invoked directly - the
option is needed to also request the unversioned
pyvenv and the Windows installer ensure that the unqualified
command is made available in those environments, and
pip can always be
invoked via the
-m switch rather than directly to avoid ambiguity on
systems with multiple Python installations.
As part of this change, the Installing Python Modules and Distributing Python Modules sections of the documentation have been completely redesigned as short getting started and FAQ documents. Most packaging documentation has now been moved out to the Python Packaging Authority maintained Python Packaging User Guide and the documentation of the individual projects.
However, as this migration is currently still incomplete, the legacy versions of those guides remaining available as Installing Python Modules (Legacy version) and Distributing Python Modules (Legacy version).
- PEP 453 – Explicit bootstrapping of pip in Python installations
- PEP written by Donald Stufft and Nick Coghlan, implemented by Donald Stufft, Nick Coghlan, Martin von Löwis and Ned Deily.
PEP 446: Newly Created File Descriptors Are Non-Inheritable
PEP 446 makes newly created file descriptors non-inheritable. In general, this is the behavior an application will want: when launching a new process, having currently open files also open in the new process can lead to all sorts of hard to find bugs, and potentially to security issues.
However, there are occasions when inheritance is desired. To support these cases, the following new functions and methods are available:
- PEP 446 – Make newly created file descriptors non-inheritable
- PEP written and implemented by Victor Stinner.
Improvements to Codec Handling
Since it was first introduced, the
codecs module has always been
intended to operate as a type-neutral dynamic encoding and decoding
system. However, its close coupling with the Python text model, especially
the type restricted convenience methods on the builtin
bytearray types, has historically obscured that
As a key step in clarifying the situation, the
codecs.decode() convenience functions are now properly documented in
Python 2.7, 3.3 and 3.4. These functions have existed in the
module (and have been covered by the regression test suite) since Python 2.4,
but were previously only discoverable through runtime introspection.
Unlike the convenience methods on
codecs convenience functions support arbitrary
codecs in both Python 2 and Python 3, rather than being limited to Unicode text
encodings (in Python 3) or
basestring conversions (in
In Python 3.4, the interpreter is able to identify the known non-text encodings provided in the standard library and direct users towards these general purpose convenience functions when appropriate:
>>> b"abcdef".decode("hex") Traceback (most recent call last): File "<stdin>", line 1, in <module> LookupError: 'hex' is not a text encoding; use codecs.decode() to handle arbitrary codecs >>> "hello".encode("rot13") Traceback (most recent call last): File "<stdin>", line 1, in <module> LookupError: 'rot13' is not a text encoding; use codecs.encode() to handle arbitrary codecs >>> open("foo.txt", encoding="hex") Traceback (most recent call last): File "<stdin>", line 1, in <module> LookupError: 'hex' is not a text encoding; use codecs.open() to handle arbitrary codecs
In a related change, whenever it is feasible without breaking backwards compatibility, exceptions raised during encoding and decoding operations are wrapped in a chained exception of the same type that mentions the name of the codec responsible for producing the error:
>>> import codecs >>> codecs.decode(b"abcdefgh", "hex") Traceback (most recent call last): File "/usr/lib/python3.4/encodings/hex_codec.py", line 20, in hex_decode return (binascii.a2b_hex(input), len(input)) binascii.Error: Non-hexadecimal digit found The above exception was the direct cause of the following exception: Traceback (most recent call last): File "<stdin>", line 1, in <module> binascii.Error: decoding with 'hex' codec failed (Error: Non-hexadecimal digit found) >>> codecs.encode("hello", "bz2") Traceback (most recent call last): File "/usr/lib/python3.4/encodings/bz2_codec.py", line 17, in bz2_encode return (bz2.compress(input), len(input)) File "/usr/lib/python3.4/bz2.py", line 498, in compress return comp.compress(data) + comp.flush() TypeError: 'str' does not support the buffer interface The above exception was the direct cause of the following exception: Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: encoding with 'bz2' codec failed (TypeError: 'str' does not support the buffer interface)
Finally, as the examples above show, these improvements have permitted the restoration of the convenience aliases for the non-Unicode codecs that were themselves restored in Python 3.2. This means that encoding binary data to and from its hexadecimal representation (for example) can now be written as:
>>> from codecs import encode, decode >>> encode(b"hello", "hex") b'68656c6c6f' >>> decode(b"68656c6c6f", "hex") b'hello'
PEP 451: A ModuleSpec Type for the Import System
PEP 451 provides an encapsulation of the information about a module that the import machinery will use to load it (that is, a module specification). This helps simplify both the import implementation and several import-related APIs. The change is also a stepping stone for several future import-related improvements.
The public-facing changes from the PEP are entirely backward-compatible. Furthermore, they should be transparent to everyone but importer authors. Key finder and loader methods have been deprecated, but they will continue working. New importers should use the new methods described in the PEP. Existing importers should be updated to implement the new methods. See the Deprecated section for a list of methods that should be replaced and their replacements.
Other Language Changes
Some smaller changes made to the core Python language are:
- Unicode database updated to UCD version 6.3.
max()now accept a default keyword-only argument that can be used to specify the value they return if the iterable they are evaluating has no elements. (Contributed by Julian Berman in issue 18111.)
- Module objects are now
__file__attributes (and related values) should now always contain absolute paths by default, with the sole exception of
__main__.__file__when a script has been executed directly using a relative path. (Contributed by Brett Cannon in issue 18416.)
- All the UTF-* codecs (except UTF-7) now reject surrogates during both
encoding and decoding unless the
surrogatepasserror handler is used, with the exception of the UTF-16 decoder (which accepts valid surrogate pairs) and the UTF-16 encoder (which produces them while encoding non-BMP characters). (Contributed by Victor Stinner, Kang-Hao (Kenny) Lu and Serhiy Storchaka in issue 12892.)
- New German EBCDIC codec
cp273. (Contributed by Michael Bierenfeld and Andrew Kuchling in issue 1097797.)
- New Ukrainian codec
cp1125. (Contributed by Serhiy Storchaka in issue 19668.)
bytearray.join() now accept arbitrary buffer objects as arguments. (Contributed by Antoine Pitrou in issue 15958.)
intconstructor now accepts any object that has an
__index__method for its base argument. (Contributed by Mark Dickinson in issue 16772.)
- Frame objects now have a
clear()method that clears all references to local variables from the frame. (Contributed by Antoine Pitrou in issue 17934.)
memoryviewis now registered as a
Sequence, and supports the
reversed()builtin. (Contributed by Nick Coghlan and Claudiu Popa in issue 18690 and issue 19078.)
- Signatures reported by
help()have been modified and improved in several cases as a result of the introduction of Argument Clinic and other changes to the
__length_hint__()is now part of the formal language specification (see PEP 424). (Contributed by Armin Ronacher in issue 16148.)
asyncio module (defined in PEP 3156) provides a standard
pluggable event loop model for Python, providing solid asynchronous IO
support in the standard library, and making it easier for other event loop
implementations to interoperate with the standard library and each other.
For Python 3.4, this module is considered a provisional API.
- PEP 3156 – Asynchronous IO Support Rebooted: the “asyncio” Module
- PEP written and implementation led by Guido van Rossum.
ensurepip module is the primary infrastructure for the
PEP 453 implementation. In the normal course of events end users will not
need to interact with this module, but it can be used to manually bootstrap
pip if the automated bootstrapping into an installation or virtual
environment was declined.
ensurepip includes a bundled copy of
pip, up-to-date as of the first
release candidate of the release of CPython with which it ships (this applies
to both maintenance releases and feature releases).
ensurepip does not
access the internet. If the installation has Internet access, after
ensurepip is run the bundled
pip can be used to upgrade
pip to a
more recent release than the bundled one. (Note that such an upgraded version
pip is considered to be a separately installed package and will not be
removed if Python is uninstalled.)
The module is named ensurepip because if called when
pip is already
installed, it does nothing. It also has an
--upgrade option that will
cause it to install the bundled copy of
pip if the existing installed
pip is older than the bundled copy.
enum module (defined in PEP 435) provides a standard
implementation of enumeration types, allowing other modules (such as
socket) to provide more informative error messages and better
debugging support by replacing opaque integer constants with backwards
compatible enumeration values.
- PEP 435 – Adding an Enum type to the Python standard library
- PEP written by Barry Warsaw, Eli Bendersky and Ethan Furman, implemented by Ethan Furman.
pathlib module offers classes representing filesystem paths
with semantics appropriate for different operating systems. Path classes are
divided between pure paths, which provide purely computational operations
without I/O, and concrete paths, which inherit from pure paths but also
provide I/O operations.
For Python 3.4, this module is considered a provisional API.
- PEP 428 – The pathlib module – object-oriented filesystem paths
- PEP written and implemented by Antoine Pitrou.
statistics module (defined in PEP 450) offers some core
statistics functionality directly in the standard library. This module
supports calculation of the mean, median, mode, variance and standard
deviation of a data series.
- PEP 450 – Adding A Statistics Module To The Standard Library
- PEP written and implemented by Steven D’Aprano
- Trace where an object was allocated
- Statistics on allocated memory blocks per filename and per line number: total size, number and average size of allocated memory blocks
- Compute the differences between two snapshots to detect memory leaks
- PEP 454 – Add