from __future__ import print_function

This is a personal choice. 2to3 handles the translation from the print statement to the print function rather well so this is an optional step. This future statement does help, though, with getting used to typing print('Hello, World') instead of print 'Hello, World'.

from __future__ import unicode_literals

Another personal choice. You can always mark what you want to be a (unicode) string with a u prefix to get the same effect. But regardless of whether you use this future statement or not, you must make sure you know exactly which Python 2 strings you want to be bytes, and which are to be strings. This means you should, at minimum mark all strings that are meant to be text strings with a u prefix if you do not use this future statement. Python 3.3 allows strings to continue to have the u prefix (it’s a no-op in that case) to make it easier for code to be source-compatible between Python 2 & 3.

Bytes literals

This is a very important one. The ability to prefix Python 2 strings that are meant to contain bytes with a b prefix help to very clearly delineate what is and is not a Python 3 string. When you run 2to3 on code, all Python 2 strings become Python 3 strings unless they are prefixed with b.

This point cannot be stressed enough: make sure you know what all of your string literals in Python 2 are meant to become in Python 3. Any string literal that should be treated as bytes should have the b prefix. Any string literal that should be Unicode/text in Python 2 should either have the u literal (supported, but ignored, in Python 3.3 and later) or you should have from __future__ import unicode_literals at the top of the file. But the key point is you should know how Python 3 will treat everyone one of your string literals and you should mark them as appropriate.

There are some differences between byte literals in Python 2 and those in Python 3 thanks to the bytes type just being an alias to str in Python 2. Probably the biggest “gotcha” is that indexing results in different values. In Python 2, the value of b'py'[1] is 'y', while in Python 3 it’s 121. You can avoid this disparity by always slicing at the size of a single element: b'py'[1:2] is 'y' in Python 2 and b'y' in Python 3 (i.e., close enough).

You cannot concatenate bytes and strings in Python 3. But since Python 2 has bytes aliased to str, it will succeed: b'a' + u'b' works in Python 2, but b'a' + 'b' in Python 3 is a TypeError. A similar issue also comes about when doing comparisons between bytes and strings.

from __future__ import absolute_import

Implicit relative imports (e.g., importing spam.bacon from within spam.eggs with the statement import bacon) does not work in Python 3. This future statement moves away from that and allows the use of explicit relative imports (e.g., from . import bacon).

In Python 2.5 you must use the __future__ statement to get to use explicit relative imports and prevent implicit ones. In Python 2.6 explicit relative imports are available without the statement, but you still want the __future__ statement to prevent implicit relative imports. In Python 2.7 the __future__ statement is not needed. In other words, unless you are only supporting Python 2.7 or a version earlier than Python 2.5, use the __future__ statement.

Mark all Unicode strings with a u prefix

While Python 2.6 has a __future__ statement to automatically cause Python 2 to treat all string literals as Unicode, Python 2.5 does not have that shortcut. This means you should go through and mark all string literals with a u prefix to turn them explicitly into Unicode strings where appropriate. That leaves all unmarked string literals to be considered byte literals in Python 3.

from __future__ import division

While the exact same outcome can be had by using the -Qnew argument to Python, using this future statement lifts the requirement that your users use the flag to get the expected behavior of division in Python 3 (e.g., 1/2 == 0.5; 1//2 == 0).

Specify when opening a file as binary

Unless you have been working on Windows, there is a chance you have not always bothered to add the b mode when opening a binary file (e.g., rb for binary reading). Under Python 3, binary files and text files are clearly distinct and mutually incompatible; see the io module for details. Therefore, you must make a decision of whether a file will be used for binary access (allowing to read and/or write bytes data) or text access (allowing to read and/or write unicode data).

Text files

Text files created using open() under Python 2 return byte strings, while under Python 3 they return unicode strings. Depending on your porting strategy, this can be an issue.

If you want text files to return unicode strings in Python 2, you have two possibilities:

• Under Python 2.6 and higher, use io.open(). Since io.open() is essentially the same function in both Python 2 and Python 3, it will help iron out any issues that might arise.
• If pre-2.6 compatibility is needed, then you should use codecs.open() instead. This will make sure that you get back unicode strings in Python 2.

Subclass object

New-style classes have been around since Python 2.2. You need to make sure you are subclassing from object to avoid odd edge cases involving method resolution order, etc. This continues to be totally valid in Python 3 (although unneeded as all classes implicitly inherit from object).

Deal With the Bytes/String Dichotomy

One of the biggest issues people have when porting code to Python 3 is handling the bytes/string dichotomy. Because Python 2 allowed the str type to hold textual data, people have over the years been rather loose in their delineation of what str instances held text compared to bytes. In Python 3 you cannot be so care-free anymore and need to properly handle the difference. The key handling this issue is to make sure that every string literal in your Python 2 code is either syntactically of functionally marked as either bytes or text data. After this is done you then need to make sure your APIs are designed to either handle a specific type or made to be properly polymorphic.

>>> b"" == ""
False

$ python3 -b
>>> b"" == ""
__main__:1: BytesWarning: Comparison between bytes and string
False

$ python3 -bb
>>> b"" == ""
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
BytesWarning: Comparison between bytes and string

Indexing bytes objects

Another potentially surprising change is the indexing behaviour of bytes objects in Python 3:

>>> b"xyz"[0]
120

Indeed, Python 3 bytes objects (as well as bytearray objects) are sequences of integers. But code converted from Python 2 will often assume that indexing a bytestring produces another bytestring, not an integer. To reconcile both behaviours, use slicing:

>>> b"xyz"[0:1]
b'x'
>>> n = 1
>>> b"xyz"[n:n+1]
b'y'

The only remaining gotcha is that an out-of-bounds slice returns an empty bytes object instead of raising IndexError:

>>> b"xyz"[3]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: index out of range
>>> b"xyz"[3:4]
b''

__str__()/__unicode__()

In Python 2, objects can specify both a string and unicode representation of themselves. In Python 3, though, there is only a string representation. This becomes an issue as people can inadvertently do things in their __str__() methods which have unpredictable results (e.g., infinite recursion if you happen to use the unicode(self).encode('utf8') idiom as the body of your __str__() method).

There are two ways to solve this issue. One is to use a custom 2to3 fixer. The blog post at http://lucumr.pocoo.org/2011/1/22/forwards-compatible-python/ specifies how to do this. That will allow 2to3 to change all instances of def __unicode(self): ... to def __str__(self): .... This does require that you define your __str__() method in Python 2 before your __unicode__() method.

The other option is to use a mixin class. This allows you to only define a __unicode__() method for your class and let the mixin derive __str__() for you (code from http://lucumr.pocoo.org/2011/1/22/forwards-compatible-python/):

import sys

class UnicodeMixin(object):

  """Mixin class to handle defining the proper __str__/__unicode__
  methods in Python 2 or 3."""

  if sys.version_info[0] >= 3: # Python 3
      def __str__(self):
          return self.__unicode__()
  else:  # Python 2
      def __str__(self):
          return self.__unicode__().encode('utf8')


class Spam(UnicodeMixin):

  def __unicode__(self):
      return u'spam-spam-bacon-spam'  # 2to3 will remove the 'u' prefix

Don’t Index on Exceptions

In Python 2, the following worked:

>>> exc = Exception(1, 2, 3)
>>> exc.args[1]
2
>>> exc[1]  # Python 2 only!
2

But in Python 3, indexing directly on an exception is an error. You need to make sure to only index on the BaseException.args attribute which is a sequence containing all arguments passed to the __init__() method.

Even better is to use the documented attributes the exception provides.

Don’t use __getslice__ & Friends

Been deprecated for a while, but Python 3 finally drops support for __getslice__(), etc. Move completely over to __getitem__() and friends.

Updating doctests

2to3 will attempt to generate fixes for doctests that it comes across. It’s not perfect, though. If you wrote a monolithic set of doctests (e.g., a single docstring containing all of your doctests), you should at least consider breaking the doctests up into smaller pieces to make it more manageable to fix. Otherwise it might very well be worth your time and effort to port your tests to unittest.

Update map for imbalanced input sequences

With Python 2, map would pad input sequences of unequal length with None values, returning a sequence as long as the longest input sequence.

With Python 3, if the input sequences to map are of unequal length, map will stop at the termination of the shortest of the sequences. For full compatibility with map from Python 2.x, also wrap the sequences in itertools.zip_longest(), e.g. map(func, *sequences) becomes list(map(func, itertools.zip_longest(*sequences))).

Manually

To manually convert source code using 2to3, you use the 2to3 script that is installed with Python 2.6 and later.:

2to3 <directory or file to convert>

This will cause 2to3 to write out a diff with all of the fixers applied for the converted source code. If you would like 2to3 to go ahead and apply the changes you can pass it the -w flag:

2to3 -w <stuff to convert>

There are other flags available to control exactly which fixers are applied, etc.

During Installation

When a user installs your project for Python 3, you can have either distutils or Distribute run 2to3 on your behalf. For distutils, use the following idiom:

try:  # Python 3
  from distutils.command.build_py import build_py_2to3 as build_py
except ImportError:  # Python 2
  from distutils.command.build_py import build_py

setup(cmdclass = {'build_py': build_py},
  # ...
)

For Distribute:

setup(use_2to3=True,
  # ...
)

This will allow you to not have to distribute a separate Python 3 version of your project. It does require, though, that when you perform development that you at least build your project and use the built Python 3 source for testing.