31.13. pickletools — Tools for pickle developers

Python 3.3

previous page next page

31.13. `pickletools` — Tools for pickle developers

This module contains various constants relating to the intimate details of the pickle module, some lengthy comments about the implementation, and a few useful functions for analyzing pickled data. The contents of this module are useful for Python core developers who are working on the pickle; ordinary users of the pickle module probably won’t find the pickletools module relevant.

31.13.1. Command line usage

New in version 3.2.

When invoked from the command line, python -m pickletools will disassemble the contents of one or more pickle files. Note that if you want to see the Python object stored in the pickle rather than the details of pickle format, you may want to use -m pickle instead. However, when the pickle file that you want to examine comes from an untrusted source, -m pickletools is a safer option because it does not execute pickle bytecode.

For example, with a tuple (1, 2) pickled in file x.pickle:

$ python -m pickle x.pickle
(1, 2)

$ python -m pickletools x.pickle
    0: \x80 PROTO      3
    2: K    BININT1    1
    4: K    BININT1    2
    6: \x86 TUPLE2
    7: q    BINPUT     0
    9: .    STOP
highest protocol among opcodes = 2

31.13.1.1. Command line options

-a, --annotate: Annotate each line with a short opcode description.

-o, --output=<file> Name of a file where the output should be written. -l, --indentlevel=<num> The number of blanks by which to indent a new MARK level. -m, --memo When multiple objects are disassembled, preserve memo between disassemblies. -p, --preamble=<preamble> When more than one pickle file are specified, print given preamble before each disassembly.

31.13.2. Programmatic Interface pickletools.dis(pickle, out=None, memo=None, indentlevel=4, annotate=0) Outputs a symbolic disassembly of the pickle to the file-like object out, defaulting to sys.stdout. pickle can be a string or a file-like object. memo can be a Python dictionary that will be used as the pickle’s memo; it can be used to perform disassemblies across multiple pickles created by the same pickler. Successive levels, indicated by MARK opcodes in the stream, are indented by indentlevel spaces. If a nonzero value is given to annotate, each opcode in the output is annotated with a short description. The value of annotate is used as a hint for the column where annotation should start. New in version 3.2: The annotate argument. pickletools.genops(pickle) Provides an iterator over all of the opcodes in a pickle, returning a sequence of (opcode, arg, pos) triples. opcode is an instance of an OpcodeInfo class; arg is the decoded value, as a Python object, of the opcode’s argument; pos is the position at which this opcode is located. pickle can be a string or a file-like object. pickletools.optimize(picklestring) Returns a new equivalent pickle string after eliminating unused PUT opcodes. The optimized pickle is shorter, takes less transmission time, requires less storage space, and unpickles more efficiently.

Navigation index modules | next | previous | Python » 3.3.3rc2 Documentation » The Python Standard Library » 31. Python Language Services » © Copyright 1990-2013, Python Software Foundation. The Python Software Foundation is a non-profit corporation. Please donate. Last updated on Nov 11, 2013. Found a bug? Created using Sphinx 1.0.7.

previous page start next page

Menu Homepage Table of contents 3.3.3rc2 Documentation Python Module Index What's New in Python What's New In Python 3.3 Summary -- Release highlights PEP 405: Virtual Environments PEP 420: Implicit Namespace Packages PEP 3118: New memoryview implementation and buffer protocol documentation Features API changes PEP 393: Flexible String Representation Functionality Performance and resource usage PEP 397: Python Launcher for Windows PEP 3151: Reworking the OS and IO exception hierarchy PEP 380: Syntax for Delegating to a Subgenerator PEP 409: Suppressing exception context PEP 414: Explicit Unicode literals PEP 3155: Qualified name for classes and functions PEP 412: Key-Sharing Dictionary PEP 362: Function Signature Object PEP 421: Adding sys.implementation SimpleNamespace Using importlib as the Implementation of Import New APIs Visible Changes Other Language Changes A Finer-Grained Import Lock Builtin functions and types New Modules faulthandler ipaddress lzma Improved Modules abc array base64 binascii bz2 codecs collections contextlib crypt curses datetime decimal Features API changes email Policy Framework Provisional Policy with New Header API Other API Changes ftplib functools gc hmac http html imaplib inspect io itertools logging math mmap multiprocessing nntplib os pdb pickle pydoc re sched select shlex shutil signal smtpd smtplib socket socketserver sqlite3 ssl stat struct subprocess sys tarfile tempfile textwrap threading time types unittest urllib webbrowser xml.etree.ElementTree zlib Optimizations Build and C API Changes Deprecated Unsupported Operating Systems Deprecated Python modules, functions and methods Deprecated functions and types of the C API Deprecated features Porting to Python 3.3 Porting Python code Porting C code Building C extensions Command Line Switch Changes What's New In Python 3.2 PEP 384: Defining a Stable ABI PEP 389: Argparse Command Line Parsing Module PEP 391: Dictionary Based Configuration for Logging PEP 3148: The concurrent.futures module PEP 3147: PYC Repository Directories PEP 3149: ABI Version Tagged .so Files PEP 3333: Python Web Server Gateway Interface v1.0.1 Other Language Changes New, Improved, and Deprecated Modules email elementtree functools itertools collections threading datetime and time math abc io reprlib logging csv contextlib decimal and fractions ftp popen select gzip and zipfile tarfile hashlib ast os shutil sqlite3 html socket ssl nntp certificates imaplib http.client unittest random poplib asyncore tempfile inspect pydoc dis dbm ctypes site sysconfig pdb configparser urllib.parse mailbox turtledemo Multi-threading Optimizations Unicode Codecs Documentation IDLE Code Repository Build and C API Changes Porting to Python 3.2 What's New In Python 3.1 PEP 372: Ordered Dictionaries PEP 378: Format Specifier for Thousands Separator Other Language Changes New, Improved, and Deprecated Modules Optimizations IDLE Build and C API Changes Porting to Python 3.1 What's New In Python 3.0 Common Stumbling Blocks Print Is A Function Views And Iterators Instead Of Lists Ordering Comparisons Integers Text Vs. Data Instead Of Unicode Vs. 8-bit Overview Of Syntax Changes New Syntax Changed Syntax Removed Syntax Changes Already Present In Python 2.6 Library Changes PEP 3101: A New Approach To String Formatting Changes To Exceptions Miscellaneous Other Changes Operators And Special Methods Builtins Build and C API Changes Performance Porting To Python 3.0 What's New in Python 2.7 The Future for Python 2.x Python 3.1 Features PEP 372: Adding an Ordered Dictionary to collections PEP 378: Format Specifier for Thousands Separator PEP 389: The argparse Module for Parsing Command Lines PEP 391: Dictionary-Based Configuration For Logging PEP 3106: Dictionary Views PEP 3137: The memoryview Object Other Language Changes Interpreter Changes Optimizations New and Improved Modules New module: importlib New module: sysconfig ttk: Themed Widgets for Tk Updated module: unittest Updated module: ElementTree 1.3 Build and C API Changes Capsules Port-Specific Changes: Windows Port-Specific Changes: Mac OS X Port-Specific Changes: FreeBSD Other Changes and Fixes Porting to Python 2.7 Acknowledgements What's New in Python 2.6 Python 3.0 Changes to the Development Process New Issue Tracker: Roundup New Documentation Format: reStructuredText Using Sphinx PEP 343: The 'with' statement Writing Context Managers The contextlib module PEP 366: Explicit Relative Imports From a Main Module PEP 370: Per-user site-packages Directory PEP 371: The multiprocessing Package PEP 3101: Advanced String Formatting PEP 3105: print As a Function PEP 3110: Exception-Handling Changes PEP 3112: Byte Literals PEP 3116: New I/O Library PEP 3118: Revised Buffer Protocol PEP 3119: Abstract Base Classes PEP 3127: Integer Literal Support and Syntax PEP 3129: Class Decorators PEP 3141: A Type Hierarchy for Numbers The fractions Module Other Language Changes Optimizations Interpreter Changes New and Improved Modules The ast module The future_builtins module The json module: JavaScript Object Notation The plistlib module: A Property-List Parser ctypes Enhancements Improved SSL Support Deprecations and Removals Build and C API Changes Port-Specific Changes: Windows Port-Specific Changes: Mac OS X Port-Specific Changes: IRIX Porting to Python 2.6 Acknowledgements What's New in Python 2.5 PEP 308: Conditional Expressions PEP 309: Partial Function Application PEP 314: Metadata for Python Software Packages v1.1 PEP 328: Absolute and Relative Imports PEP 338: Executing Modules as Scripts PEP 341: Unified try/except/finally PEP 342: New Generator Features PEP 343: The 'with' statement Writing Context Managers The contextlib module PEP 352: Exceptions as New-Style Classes PEP 353: Using ssize_t as the index type PEP 357: The '__index__' method Other Language Changes Interactive Interpreter Changes Optimizations New, Improved, and Removed Modules The ctypes package The ElementTree package The hashlib package The sqlite3 package The wsgiref package Build and C API Changes Port-Specific Changes Porting to Python 2.5 Acknowledgements What's New in Python 2.4 PEP 218: Built-In Set Objects PEP 237: Unifying Long Integers and Integers PEP 289: Generator Expressions PEP 292: Simpler String Substitutions PEP 318: Decorators for Functions and Methods PEP 322: Reverse Iteration PEP 324: New subprocess Module PEP 327: Decimal Data Type Why is Decimal needed? The Decimal type The Context type PEP 328: Multi-line Imports PEP 331: Locale-Independent Float/String Conversions Other Language Changes Optimizations New, Improved, and Deprecated Modules cookielib doctest Build and C API Changes Port-Specific Changes Porting to Python 2.4 Acknowledgements What's New in Python 2.3 PEP 218: A Standard Set Datatype PEP 255: Simple Generators PEP 263: Source Code Encodings PEP 273: Importing Modules from ZIP Archives PEP 277: Unicode file name support for Windows NT PEP 278: Universal Newline Support PEP 279: enumerate() PEP 282: The logging Package PEP 285: A Boolean Type PEP 293: Codec Error Handling Callbacks PEP 301: Package Index and Metadata for Distutils PEP 302: New Import Hooks PEP 305: Comma-separated Files PEP 307: Pickle Enhancements Extended Slices Other Language Changes String Changes Optimizations New, Improved, and Deprecated Modules Date/Time Type The optparse Module Pymalloc: A Specialized Object Allocator Build and C API Changes Port-Specific Changes Other Changes and Fixes Porting to Python 2.3 Acknowledgements What's New in Python 2.2 Introduction PEPs 252 and 253: Type and Class Changes Old and New Classes Descriptors Multiple Inheritance: The Diamond Rule Attribute Access Related Links PEP 234: Iterators PEP 255: Simple Generators PEP 237: Unifying Long Integers and Integers PEP 238: Changing the Division Operator Unicode Changes PEP 227: Nested Scopes New and Improved Modules Interpreter Changes and Fixes Other Changes and Fixes Acknowledgements What's New in Python 2.1 Introduction PEP 227: Nested Scopes PEP 236: __future__ Directives PEP 207: Rich Comparisons PEP 230: Warning Framework PEP 229: New Build System PEP 205: Weak References PEP 232: Function Attributes PEP 235: Importing Modules on Case-Insensitive Platforms PEP 217: Interactive Display Hook PEP 208: New Coercion Model PEP 241: Metadata in Python Packages New and Improved Modules Other Changes and Fixes Acknowledgements What's New in Python 2.0 Introduction What About Python 1.6? New Development Process Unicode List Comprehensions Augmented Assignment String Methods Garbage Collection of Cycles Other Core Changes Minor Language Changes Changes to Built-in Functions Porting to 2.0 Extending/Embedding Changes Distutils: Making Modules Easy to Install XML Modules SAX2 Support DOM Support Relationship to PyXML Module changes New modules IDLE Improvements Deleted and Deprecated Modules Acknowledgements Changelog Python 3.3.3 release candidate 2? Library Tests Build Python 3.3.3 release candidate 1? Core and Builtins Library C API IDLE Tests Documentation Tools/Demos Build Python 3.3.2? Core and Builtins Library IDLE Tests Documentation Build Python 3.3.1? Build IDLE Python 3.3.1 release candidate 1? Core and Builtins Library IDLE Tests Build C-API Documentation Tools/Demos Python 3.3.0? Core and Builtins Extension Modules Python 3.3.0 Release Candidate 3? Core and Builtins Library Extension Modules Python 3.3.0 Release Candidate 2? Core and Builtins Library Build Documentation Python 3.3.0 Release Candidate 1? Core and Builtins Library IDLE Documentation Tests Build Python 3.3.0 Beta 2? Core and Builtins Library C API Extension Modules IDLE Tools/Demos Documentation Tests Build Python 3.3.0 Beta 1? Core and Builtins Library C-API Extension Modules IDLE Documentation Tests Build Python 3.3.0 Alpha 4? Core and Builtins Library IDLE Tools/Demos Build C-API Documentation Python 3.3.0 Alpha 3? Core and Builtins Library Build Tests IDLE Tools / Demos C-API Python 3.3.0 Alpha 2? Core and Builtins Library Build Documentation Extension Modules Tests Python 3.3.0 Alpha 1? Core and Builtins Library Build IDLE Tools/Demos Extension Modules Tests C-API Documentation The Python Tutorial Whetting Your Appetite Using the Python Interpreter Invoking the Interpreter Argument Passing Interactive Mode The Interpreter and Its Environment Error Handling Executable Python Scripts Source Code Encoding The Interactive Startup File The Customization Modules An Informal Introduction to Python Using Python as a Calculator Numbers Strings Lists First Steps Towards Programming More Control Flow Tools if Statements for Statements The range() Function break and continue Statements, and else Clauses on Loops pass Statements Defining Functions More on Defining Functions Default Argument Values Keyword Arguments Arbitrary Argument Lists Unpacking Argument Lists Lambda Expressions Documentation Strings Function Annotations Intermezzo: Coding Style Data Structures More on Lists Using Lists as Stacks Using Lists as Queues List Comprehensions Nested List Comprehensions The del statement Tuples and Sequences Sets Dictionaries Looping Techniques More on Conditions Comparing Sequences and Other Types Modules More on Modules Executing modules as scripts The Module Search Path "Compiled" Python files Standard Modules The dir() Function Packages Importing * From a Package Intra-package References Packages in Multiple Directories Input and Output Fancier Output Formatting Old string formatting Reading and Writing Files Methods of File Objects The pickle Module Errors and Exceptions Syntax Errors Exceptions Handling Exceptions Raising Exceptions User-defined Exceptions Defining Clean-up Actions Predefined Clean-up Actions Classes A Word About Names and Objects Python Scopes and Namespaces Scopes and Namespaces Example A First Look at Classes Class Definition Syntax Class Objects Instance Objects Method Objects Random Remarks Inheritance Multiple Inheritance Private Variables Odds and Ends Exceptions Are Classes Too Iterators Generators Generator Expressions Brief Tour of the Standard Library Operating System Interface File Wildcards Command Line Arguments Error Output Redirection and Program Termination String Pattern Matching Mathematics Internet Access Dates and Times Data Compression Performance Measurement Quality Control Batteries Included Brief Tour of the Standard Library -- Part II Output Formatting Templating Working with Binary Data Record Layouts Multi-threading Logging Weak References Tools for Working with Lists Decimal Floating Point Arithmetic What Now? Interactive Input Editing and History Substitution Line Editing History Substitution Key Bindings Alternatives to the Interactive Interpreter Floating Point Arithmetic: Issues and Limitations Representation Error Python Setup and Usage Command line and environment Command line Interface options Generic options Miscellaneous options Options you shouldn't use Environment variables Debug-mode variables Using Python on Unix platforms Getting and installing the latest version of Python On Linux On FreeBSD and OpenBSD On OpenSolaris Building Python Python-related paths and files Miscellaneous Editors Using Python on Windows Installing Python Alternative bundles Configuring Python Excursus: Setting environment variables Finding the Python executable Finding modules Executing scripts Executing scripts without the Python launcher Python Launcher for Windows Getting started From the command-line From a script From file associations Shebang Lines Arguments in shebang lines Customization Customization via INI files Customizing default Python versions Diagnostics Additional modules PyWin32 Py2exe WConio Compiling Python on Windows Other resources Using Python on a Macintosh Getting and Installing MacPython How to run a Python script Running scripts with a GUI Configuration The IDE Installing Additional Python Packages GUI Programming on the Mac Distributing Python Applications on the Mac Other Resources Additional Tools and Scripts pyvenv - Creating virtual environments The Python Language Reference Introduction Alternate Implementations Notation Lexical analysis Line structure Logical lines Physical lines Comments Encoding declarations Explicit line joining Implicit line joining Blank lines Indentation Whitespace between tokens Other tokens Identifiers and keywords Keywords Reserved classes of identifiers Literals String and Bytes literals String literal concatenation Numeric literals Integer literals Floating point literals Imaginary literals Operators Delimiters Data model Objects, values and types The standard type hierarchy Special method names Basic customization Customizing attribute access Implementing Descriptors Invoking Descriptors __slots__ Notes on using __slots__ Customizing class creation Determining the appropriate metaclass Preparing the class namespace Executing the class body Creating the class object Metaclass example Customizing instance and subclass checks Emulating callable objects Emulating container types Emulating numeric types With Statement Context Managers Special method lookup Execution model Naming and binding Interaction with dynamic features Exceptions The import system importlib Packages Regular packages Namespace packages Searching The module cache Finders and loaders Import hooks The meta path Loaders Module reprs module.__path__ The Path Based Finder Path entry finders Path entry finder protocol Replacing the standard import system Open issues References Expressions Arithmetic conversions Atoms Identifiers (Names) Literals Parenthesized forms Displays for lists, sets and dictionaries List displays Set displays Dictionary displays Generator expressions Yield expressions Generator-iterator methods Examples Primaries Attribute references Subscriptions Slicings Calls The power operator Unary arithmetic and bitwise operations Binary arithmetic operations Shifting operations Binary bitwise operations Comparisons Boolean operations Conditional expressions Lambdas Expression lists Evaluation order Operator precedence Simple statements Expression statements Assignment statements Augmented assignment statements The assert statement The pass statement The del statement The return statement The yield statement The raise statement The break statement The continue statement The import statement Future statements The global statement The nonlocal statement Compound statements The if statement The while statement The for statement The try statement The with statement Function definitions Class definitions Top-level components Complete Python programs File input Interactive input Expression input Full Grammar specification The Python Standard Library Introduction Built-in Functions Built-in Constants Constants added by the site module Built-in Types Truth Value Testing Boolean Operations --- and, or, not Comparisons Numeric Types --- int, float, complex Bitwise Operations on Integer Types Additional Methods on Integer Types Additional Methods on Float Hashing of numeric types Iterator Types Generator Types Sequence Types --- list, tuple, range Common Sequence Operations Immutable Sequence Types Mutable Sequence Types Lists Tuples Ranges Text Sequence Type --- str String Methods printf-style String Formatting Binary Sequence Types --- bytes, bytearray, memoryview Bytes Bytearray Objects Bytes and Bytearray Operations Memory Views Set Types --- set, frozenset Mapping Types --- dict Dictionary view objects Context Manager Types Other Built-in Types Modules Classes and Class Instances Functions Methods Code Objects Type Objects The Null Object The Ellipsis Object The NotImplemented Object Boolean Values Internal Objects Special Attributes Built-in Exceptions Base classes Concrete exceptions OS exceptions Warnings Exception hierarchy Text Processing Services string --- Common string operations String constants String Formatting Format String Syntax Format Specification Mini-Language Format examples Template strings Helper functions re --- Regular expression operations Regular Expression Syntax Module Contents Regular Expression Objects Match Objects Regular Expression Examples Checking for a Pair Simulating scanf() search() vs. match() Making a Phonebook Text Munging Finding all Adverbs Finding all Adverbs and their Positions Raw String Notation Writing a Tokenizer difflib --- Helpers for computing deltas SequenceMatcher Objects SequenceMatcher Examples Differ Objects Differ Example A command-line interface to difflib textwrap --- Text wrapping and filling unicodedata --- Unicode Database stringprep --- Internet String Preparation readline --- GNU readline interface Example rlcompleter --- Completion function for GNU readline Completer Objects Binary Data Services struct --- Interpret bytes as packed binary data Functions and Exceptions Format Strings Byte Order, Size, and Alignment Format Characters Examples Classes codecs --- Codec registry and base classes Codec Base Classes Codec Objects IncrementalEncoder Objects IncrementalDecoder Objects StreamWriter Objects StreamReader Objects StreamReaderWriter Objects StreamRecoder Objects Encodings and Unicode Standard Encodings Python Specific Encodings encodings.idna --- Internationalized Domain Names in Applications encodings.mbcs --- Windows ANSI codepage encodings.utf_8_sig --- UTF-8 codec with BOM signature Data Types datetime --- Basic date and time types Available Types timedelta Objects date Objects datetime Objects time Objects tzinfo Objects timezone Objects strftime() and strptime() Behavior calendar --- General calendar-related functions collections --- Container datatypes ChainMap objects ChainMap Examples and Recipes Counter objects deque objects deque Recipes defaultdict objects defaultdict Examples namedtuple() Factory Function for Tuples with Named Fields OrderedDict objects OrderedDict Examples and Recipes UserDict objects UserList objects UserString objects collections.abc --- Abstract Base Classes for Containers Collections Abstract Base Classes heapq --- Heap queue algorithm Basic Examples Priority Queue Implementation Notes Theory bisect --- Array bisection algorithm Searching Sorted Lists Other Examples array --- Efficient arrays of numeric values weakref --- Weak references Weak Reference Objects Example types --- Dynamic type creation and names for built-in types Dynamic Type Creation Standard Interpreter Types copy --- Shallow and deep copy operations pprint --- Data pretty printer PrettyPrinter Objects Example reprlib --- Alternate repr() implementation Repr Objects Subclassing Repr Objects Numeric and Mathematical Modules numbers --- Numeric abstract base classes The numeric tower Notes for type implementors Adding More Numeric ABCs Implementing the arithmetic operations math --- Mathematical functions Number-theoretic and representation functions Power and logarithmic functions Trigonometric functions Angular conversion Hyperbolic functions Special functions Constants cmath --- Mathematical functions for complex numbers Conversions to and from polar coordinates Power and logarithmic functions Trigonometric functions Hyperbolic functions Classification functions Constants decimal --- Decimal fixed point and floating point arithmetic Quick-start Tutorial Decimal objects Logical operands Context objects Constants Rounding modes Signals Floating Point Notes Mitigating round-off error with increased precision Special values Working with threads Recipes Decimal FAQ fractions --- Rational numbers random --- Generate pseudo-random numbers Notes on Reproducibility Examples and Recipes Functional Programming Modules itertools --- Functions creating iterators for efficient looping Itertool functions Itertools Recipes functools --- Higher-order functions and operations on callable objects partial Objects operator --- Standard operators as functions Mapping Operators to Functions Inplace Operators File and Directory Access os.path --- Common pathname manipulations fileinput --- Iterate over lines from multiple input streams stat --- Interpreting stat() results filecmp --- File and Directory Comparisons The dircmp class tempfile --- Generate temporary files and directories Examples glob --- Unix style pathname pattern expansion fnmatch --- Unix filename pattern matching linecache --- Random access to text lines shutil --- High-level file operations Directory and files operations copytree example Archiving operations Archiving example Querying the size of the output terminal macpath --- Mac OS 9 path manipulation functions Data Persistence pickle --- Python object serialization Relationship to other Python modules Data stream format Module Interface What can be pickled and unpickled? Pickling Class Instances Persistence of External Objects Dispatch Tables Handling Stateful Objects Restricting Globals Examples copyreg --- Register pickle support functions Example shelve --- Python object persistence Restrictions Example marshal --- Internal Python object serialization dbm --- Interfaces to Unix "databases" dbm.gnu --- GNU's reinterpretation of dbm dbm.ndbm --- Interface based on ndbm dbm.dumb --- Portable DBM implementation sqlite3 --- DB-API 2.0 interface for SQLite databases Module functions and constants Connection Objects Cursor Objects Row Objects SQLite and Python types Introduction Using adapters to store additional Python types in SQLite databases Letting your object adapt itself Registering an adapter callable Converting SQLite values to custom Python types Default adapters and converters Controlling Transactions Using sqlite3 efficiently Using shortcut methods Accessing columns by name instead of by index Using the connection as a context manager Common issues Multithreading Data Compression and Archiving zlib --- Compression compatible with gzip gzip --- Support for gzip files Examples of usage bz2 --- Support for bzip2 compression (De)compression of files Incremental (de)compression One-shot (de)compression lzma --- Compression using the LZMA algorithm Reading and writing compressed files Compressing and decompressing data in memory Miscellaneous Specifying custom filter chains Examples zipfile --- Work with ZIP archives ZipFile Objects PyZipFile Objects ZipInfo Objects tarfile --- Read and write tar archive files TarFile Objects TarInfo Objects Examples Supported tar formats Unicode issues File Formats csv --- CSV File Reading and Writing Module Contents Dialects and Formatting Parameters Reader Objects Writer Objects Examples configparser --- Configuration file parser Quick Start Supported Datatypes Fallback Values Supported INI File Structure Interpolation of values Mapping Protocol Access Customizing Parser Behaviour Legacy API Examples ConfigParser Objects RawConfigParser Objects Exceptions netrc --- netrc file processing netrc Objects xdrlib --- Encode and decode XDR data Packer Objects Unpacker Objects Exceptions plistlib --- Generate and parse Mac OS X .plist files Examples Cryptographic Services hashlib --- Secure hashes and message digests hmac --- Keyed-Hashing for Message Authentication Generic Operating System Services os --- Miscellaneous operating system interfaces File Names, Command Line Arguments, and Environment Variables Process Parameters File Object Creation File Descriptor Operations open() flag constants Querying the size of a terminal Files and Directories Linux extended attributes Process Management Interface to the scheduler Miscellaneous System Information Miscellaneous Functions io --- Core tools for working with streams Overview Text I/O Binary I/O Raw I/O High-level Module Interface In-memory streams Class hierarchy I/O Base Classes Raw File I/O Buffered Streams Text I/O Performance Binary I/O Text I/O Multi-threading Reentrancy time --- Time access and conversions argparse --- Parser for command-line options, arguments and sub-commands Example Creating a parser Adding arguments Parsing arguments ArgumentParser objects prog usage description epilog parents formatter_class prefix_chars fromfile_prefix_chars argument_default conflict_handler add_help The add_argument() method name or flags action nargs const default type choices required help metavar dest The parse_args() method Option value syntax Invalid arguments Arguments containing - Argument abbreviations Beyond sys.argv The Namespace object Other utilities Sub-commands FileType objects Argument groups Mutual exclusion Parser defaults Printing help Partial parsing Customizing file parsing Exiting methods Upgrading optparse code optparse --- Parser for command line options Background Terminology What are options for? What are positional arguments for? Tutorial Understanding option actions The store action Handling boolean (flag) options Other actions Default values Generating help Grouping Options Printing a version string How optparse handles errors Putting it all together Reference Guide Creating the parser Populating the parser Defining options Option attributes Standard option actions Standard option types Parsing arguments Querying and manipulating your option parser Conflicts between options Cleanup Other methods Option Callbacks Defining a callback option How callbacks are called Raising errors in a callback Callback example 1: trivial callback Callback example 2: check option order Callback example 3: check option order (generalized) Callback example 4: check arbitrary condition Callback example 5: fixed arguments Callback example 6: variable arguments Extending optparse Adding new types Adding new actions getopt --- C-style parser for command line options logging --- Logging facility for Python Logger Objects Handler Objects Formatter Objects Filter Objects LogRecord Objects LogRecord attributes LoggerAdapter Objects Thread Safety Module-Level Functions Module-Level Attributes Integration with the warnings module logging.config --- Logging configuration Configuration functions Configuration dictionary schema Dictionary Schema Details Incremental Configuration Object connections User-defined objects Access to external objects Access to internal objects Import resolution and custom importers Configuration file format logging.handlers --- Logging handlers StreamHandler FileHandler NullHandler WatchedFileHandler BaseRotatingHandler RotatingFileHandler TimedRotatingFileHandler SocketHandler DatagramHandler SysLogHandler NTEventLogHandler SMTPHandler MemoryHandler HTTPHandler QueueHandler QueueListener getpass --- Portable password input curses --- Terminal handling for character-cell displays Functions Window Objects Constants curses.textpad --- Text input widget for curses programs Textbox objects curses.ascii --- Utilities for ASCII characters curses.panel --- A panel stack extension for curses Functions Panel Objects platform --- Access to underlying platform's identifying data Cross Platform Java Platform Windows Platform Win95/98 specific Mac OS Platform Unix Platforms errno --- Standard errno system symbols ctypes --- A foreign function library for Python ctypes tutorial Loading dynamic link libraries Accessing functions from loaded dlls Calling functions Fundamental data types Calling functions, continued Calling functions with your own custom data types Specifying the required argument types (function prototypes) Return types Passing pointers (or: passing parameters by reference) Structures and unions Structure/union alignment and byte order Bit fields in structures and unions Arrays Pointers Type conversions Incomplete Types Callback functions Accessing values exported from dlls Surprises Variable-sized data types ctypes reference Finding shared libraries Loading shared libraries Foreign functions Function prototypes Utility functions Data types Fundamental data types Structured data types Arrays and pointers Concurrent Execution threading --- Thread-based parallelism Thread-Local Data Thread Objects Lock Objects RLock Objects Condition Objects Semaphore Objects Semaphore Example Event Objects Timer Objects Barrier Objects Using locks, conditions, and semaphores in the with statement multiprocessing --- Process-based parallelism Introduction The Process class Exchanging objects between processes Synchronization between processes Sharing state between processes Using a pool of workers Reference Process and exceptions Pipes and Queues Miscellaneous Connection Objects Synchronization primitives Shared ctypes Objects The multiprocessing.sharedctypes module Managers Namespace objects Customized managers Using a remote manager Proxy Objects Cleanup Process Pools Listeners and Clients Address Formats Authentication keys Logging The multiprocessing.dummy module Programming guidelines All platforms Windows Examples The concurrent package concurrent.futures --- Launching parallel tasks Executor Objects ThreadPoolExecutor ThreadPoolExecutor Example ProcessPoolExecutor ProcessPoolExecutor Example Future Objects Module Functions Exception classes subprocess --- Subprocess management Using the subprocess Module Frequently Used Arguments Popen Constructor Exceptions Security Popen Objects Windows Popen Helpers Constants Replacing Older Functions with the subprocess Module Replacing /bin/sh shell backquote Replacing shell pipeline Replacing os.system() Replacing the os.spawn family Replacing os.popen(), os.popen2(), os.popen3() Replacing functions from the popen2 module Legacy Shell Invocation Functions Notes Converting an argument sequence to a string on Windows sched --- Event scheduler Scheduler Objects queue --- A synchronized queue class Queue Objects select --- Waiting for I/O completion /dev/poll Polling Objects Edge and Level Trigger Polling (epoll) Objects Polling Objects Kqueue Objects Kevent Objects dummy_threading --- Drop-in replacement for the threading module _thread --- Low-level threading API _dummy_thread --- Drop-in replacement for the _thread module Interprocess Communication and Networking socket --- Low-level networking interface Socket families Module contents Socket Objects Notes on socket timeouts Timeouts and the connect method Timeouts and the accept method Example ssl --- TLS/SSL wrapper for socket objects Functions, Constants, and Exceptions Socket creation Random generation Certificate handling Constants SSL Sockets SSL Contexts Certificates Certificate chains CA certificates Combined key and certificate Self-signed certificates Examples Testing for SSL support Client-side operation Server-side operation Notes on non-blocking sockets Security considerations Verifying certificates Protocol versions Cipher selection asyncore --- Asynchronous socket handler asyncore Example basic HTTP client asyncore Example basic echo server asynchat --- Asynchronous socket command/response handler asynchat - Auxiliary Classes asynchat Example signal --- Set handlers for asynchronous events General rules Execution of Python signal handlers Signals and threads Module contents Example mmap --- Memory-mapped file support Internet Data Handling email --- An email and MIME handling package email.message: Representing an email message email.parser: Parsing email messages FeedParser API Parser class API Additional notes email.generator: Generating MIME documents email.policy: Policy Objects email.headerregistry: Custom Header Objects email.mime: Creating email and MIME objects from scratch email.header: Internationalized headers email.charset: Representing character sets email.encoders: Encoders email.errors: Exception and Defect classes email.utils: Miscellaneous utilities email.iterators: Iterators email: Examples Package History Differences from mimelib json --- JSON encoder and decoder Basic Usage Encoders and Decoders Standard Compliance Character Encodings Top-level Non-Object, Non-Array Values Infinite and NaN Number Values Repeated Names Within an Object mailcap --- Mailcap file handling mailbox --- Manipulate mailboxes in various formats Mailbox objects Maildir mbox MH Babyl MMDF Message objects MaildirMessage mboxMessage MHMessage BabylMessage MMDFMessage Exceptions Examples mimetypes --- Map filenames to MIME types MimeTypes Objects base64 --- RFC 3548: Base16, Base32, Base64 Data Encodings binhex --- Encode and decode binhex4 files Notes binascii --- Convert between binary and ASCII quopri --- Encode and decode MIME quoted-printable data uu --- Encode and decode uuencode files Structured Markup Processing Tools html --- HyperText Markup Language support html.parser --- Simple HTML and XHTML parser Example HTML Parser Application HTMLParser Methods Examples html.entities --- Definitions of HTML general entities XML Processing Modules XML vulnerabilities defused packages xml.etree.ElementTree --- The ElementTree XML API Tutorial XML tree and elements Parsing XML Finding interesting elements Modifying an XML File Building XML documents Additional resources XPath support Example Supported XPath syntax Reference Functions Element Objects ElementTree Objects QName Objects TreeBuilder Objects XMLParser Objects Exceptions xml.dom --- The Document Object Model API Module Contents Objects in the DOM DOMImplementation Objects Node Objects NodeList Objects DocumentType Objects Document Objects Element Objects Attr Objects NamedNodeMap Objects Comment Objects Text and CDATASection Objects ProcessingInstruction Objects Exceptions Conformance Type Mapping Accessor Methods xml.dom.minidom --- Minimal DOM implementation DOM Objects DOM Example minidom and the DOM standard xml.dom.pulldom --- Support for building partial DOM trees DOMEventStream Objects xml.sax --- Support for SAX2 parsers SAXException Objects xml.sax.handler --- Base classes for SAX handlers ContentHandler Objects DTDHandler Objects EntityResolver Objects ErrorHandler Objects xml.sax.saxutils --- SAX Utilities xml.sax.xmlreader --- Interface for XML parsers XMLReader Objects IncrementalParser Objects Locator Objects InputSource Objects The Attributes Interface The AttributesNS Interface xml.parsers.expat --- Fast XML parsing using Expat XMLParser Objects ExpatError Exceptions Example Content Model Descriptions Expat error constants Internet Protocols and Support webbrowser --- Convenient Web-browser controller Browser Controller Objects cgi --- Common Gateway Interface support Introduction Using the cgi module Higher Level Interface Functions Caring about security Installing your CGI script on a Unix system Testing your CGI script Debugging CGI scripts Common problems and solutions cgitb --- Traceback manager for CGI scripts wsgiref --- WSGI Utilities and Reference Implementation wsgiref.util -- WSGI environment utilities wsgiref.headers -- WSGI response header tools wsgiref.simple_server -- a simple WSGI HTTP server wsgiref.validate --- WSGI conformance checker wsgiref.handlers -- server/gateway base classes Examples urllib --- URL handling modules urllib.request --- Extensible library for opening URLs Request Objects OpenerDirector Objects BaseHandler Objects HTTPRedirectHandler Objects HTTPCookieProcessor Objects ProxyHandler Objects HTTPPasswordMgr Objects AbstractBasicAuthHandler Objects HTTPBasicAuthHandler Objects ProxyBasicAuthHandler Objects AbstractDigestAuthHandler Objects HTTPDigestAuthHandler Objects ProxyDigestAuthHandler Objects HTTPHandler Objects HTTPSHandler Objects FileHandler Objects FTPHandler Objects CacheFTPHandler Objects UnknownHandler Objects HTTPErrorProcessor Objects Examples Legacy interface urllib.request Restrictions urllib.response --- Response classes used by urllib urllib.parse --- Parse URLs into components URL Parsing Parsing ASCII Encoded Bytes Structured Parse Results URL Quoting urllib.error --- Exception classes raised by urllib.request urllib.robotparser --- Parser for robots.txt http --- HTTP modules http.client --- HTTP protocol client HTTPConnection Objects HTTPResponse Objects Examples HTTPMessage Objects ftplib --- FTP protocol client FTP Objects FTP_TLS Objects poplib --- POP3 protocol client POP3 Objects POP3 Example imaplib --- IMAP4 protocol client IMAP4 Objects IMAP4 Example nntplib --- NNTP protocol client NNTP Objects Attributes Methods Utility functions smtplib --- SMTP protocol client SMTP Objects SMTP Example smtpd --- SMTP Server SMTPServer Objects DebuggingServer Objects PureProxy Objects MailmanProxy Objects SMTPChannel Objects telnetlib --- Telnet client Telnet Objects Telnet Example uuid --- UUID objects according to RFC 4122 Example socketserver --- A framework for network servers Server Creation Notes Server Objects RequestHandler Objects Examples socketserver.TCPServer Example socketserver.UDPServer Example Asynchronous Mixins http.server --- HTTP servers http.cookies --- HTTP state management Cookie Objects Morsel Objects Example http.cookiejar --- Cookie handling for HTTP clients CookieJar and FileCookieJar Objects FileCookieJar subclasses and co-operation with web browsers CookiePolicy Objects DefaultCookiePolicy Objects Cookie Objects Examples xmlrpc --- XMLRPC server and client modules xmlrpc.client --- XML-RPC client access ServerProxy Objects DateTime Objects Binary Objects Fault Objects ProtocolError Objects MultiCall Objects Convenience Functions Example of Client Usage Example of Client and Server Usage xmlrpc.server --- Basic XML-RPC servers SimpleXMLRPCServer Objects SimpleXMLRPCServer Example CGIXMLRPCRequestHandler Documenting XMLRPC server DocXMLRPCServer Objects DocCGIXMLRPCRequestHandler ipaddress --- IPv4/IPv6 manipulation library Convenience factory functions IP Addresses Address objects Conversion to Strings and Integers Operators Comparison operators Arithmetic operators IP Network definitions Prefix, net mask and host mask Network objects Operators Logical operators Iteration Networks as containers of addresses Interface objects Other Module Level Functions Custom Exceptions Multimedia Services audioop --- Manipulate raw audio data aifc --- Read and write AIFF and AIFC files sunau --- Read and write Sun AU files AU_read Objects AU_write Objects wave --- Read and write WAV files Wave_read Objects Wave_write Objects chunk --- Read IFF chunked data colorsys --- Conversions between color systems imghdr --- Determine the type of an image sndhdr --- Determine type of sound file ossaudiodev --- Access to OSS-compatible audio devices Audio Device Objects Mixer Device Objects Internationalization gettext --- Multilingual internationalization services GNU gettext API Class-based API The NullTranslations class The GNUTranslations class Solaris message catalog support The Catalog constructor Internationalizing your programs and modules Localizing your module Localizing your application Changing languages on the fly Deferred translations Acknowledgements locale --- Internationalization services Background, details, hints, tips and caveats For extension writers and programs that embed Python Access to message catalogs Program Frameworks turtle --- Turtle graphics Introduction Overview of available Turtle and Screen methods Turtle methods Methods of TurtleScreen/Screen Methods of RawTurtle/Turtle and corresponding functions Turtle motion Tell Turtle's state Settings for measurement Pen control Drawing state Color control Filling More drawing control Turtle state Visibility Appearance Using events Special Turtle methods Compound shapes Methods of TurtleScreen/Screen and corresponding functions Window control Animation control Using screen events Input methods Settings and special methods Methods specific to Screen, not inherited from TurtleScreen Public classes Help and configuration How to use help Translation of docstrings into different languages How to configure Screen and Turtles Demo scripts Changes since Python 2.6 Changes since Python 3.0 cmd --- Support for line-oriented command interpreters Cmd Objects Cmd Example shlex --- Simple lexical analysis shlex Objects Parsing Rules Graphical User Interfaces with Tk tkinter --- Python interface to Tcl/Tk Tkinter Modules Tkinter Life Preserver How To Use This Section A Simple Hello World Program A (Very) Quick Look at Tcl/Tk Mapping Basic Tk into Tkinter How Tk and Tkinter are Related Handy Reference Setting Options The Packer Packer Options Coupling Widget Variables The Window Manager Tk Option Data Types Bindings and Events The index Parameter Images tkinter.ttk --- Tk themed widgets Using Ttk Ttk Widgets Widget Standard Options Scrollable Widget Options Label Options Compatibility Options Widget States ttk.Widget Combobox Options Virtual events ttk.Combobox Notebook Options Tab Options Tab Identifiers Virtual Events ttk.Notebook Progressbar Options ttk.Progressbar Separator Options Sizegrip Platform-specific notes Bugs Treeview Options Item Options Tag Options Column Identifiers Virtual Events ttk.Treeview Ttk Styling Layouts tkinter.tix --- Extension widgets for Tk Using Tix Tix Widgets Basic Widgets File Selectors Hierarchical ListBox Tabular ListBox Manager Widgets Image Types Miscellaneous Widgets Form Geometry Manager Tix Commands tkinter.scrolledtext --- Scrolled Text Widget IDLE Menus File menu Edit menu Windows menu Debug menu Edit context menu Shell context menu Basic editing and navigation Automatic indentation Python Shell window Syntax colors Startup Command line usage Other Graphical User Interface Packages Development Tools pydoc --- Documentation generator and online help system doctest --- Test interactive Python examples Simple Usage: Checking Examples in Docstrings Simple Usage: Checking Examples in a Text File How It Works Which Docstrings Are Examined? How are Docstring Examples Recognized? What's the Execution Context? What About Exceptions? Option Flags Directives Warnings Basic API Unittest API Advanced API DocTest Objects Example Objects DocTestFinder objects DocTestParser objects DocTestRunner objects OutputChecker objects Debugging Soapbox unittest --- Unit testing framework Basic example Command-Line Interface Command-line options Test Discovery Organizing test code Re-using old test code Skipping tests and expected failures Classes and functions Test cases Deprecated aliases Grouping tests Loading and running tests load_tests Protocol Class and Module Fixtures setUpClass and tearDownClass setUpModule and tearDownModule Signal Handling unittest.mock --- mock object library Quick Guide The Mock Class Calling Deleting Attributes Mock names and the name attribute Attaching Mocks as Attributes The patchers patch patch.object patch.dict patch.multiple patch methods: start and stop TEST_PREFIX Nesting Patch Decorators Where to patch Patching Descriptors and Proxy Objects MagicMock and magic method support Mocking Magic Methods Magic Mock Helpers sentinel DEFAULT call create_autospec ANY FILTER_DIR mock_open Autospeccing unittest.mock --- getting started Using Mock Mock Patching Methods Mock for Method Calls on an Object Mocking Classes Naming your mocks Tracking all Calls Setting Return Values and Attributes Raising exceptions with mocks Side effect functions and iterables Creating a Mock from an Existing Object Patch Decorators Further Examples Mocking chained calls Partial mocking Mocking a Generator Method Applying the same patch to every test method Mocking Unbound Methods Checking multiple calls with mock Coping with mutable arguments Nesting Patches Mocking a dictionary with MagicMock Mock subclasses and their attributes Mocking imports with patch.dict Tracking order of calls and less verbose call assertions More complex argument matching 2to3 - Automated Python 2 to 3 code translation Using 2to3 Fixers lib2to3 - 2to3's library test --- Regression tests package for Python Writing Unit Tests for the test package Running tests using the command-line interface test.support --- Utilities for the Python test suite venv --- Creation of virtual environments Creating virtual environments API An example of extending EnvBuilder Debugging and Profiling bdb --- Debugger framework faulthandler --- Dump the Python traceback Dump the traceback Fault handler state Dump the tracebacks after a timeout Dump the traceback on a user signal File descriptor issue Example pdb --- The Python Debugger Debugger Commands The Python Profilers Introduction to the profilers Instant User's Manual profile and cProfile Module Reference The Stats Class What Is Deterministic Profiling? Limitations Calibration Using a custom timer timeit --- Measure execution time of small code snippets Basic Examples Python Interface Command-Line Interface Examples trace --- Trace or track Python statement execution Command-Line Usage Main options Modifiers Filters Programmatic Interface Python Runtime Services sys --- System-specific parameters and functions sysconfig --- Provide access to Python's configuration information Configuration variables Installation paths Other functions Using sysconfig as a script builtins --- Built-in objects __main__ --- Top-level script environment warnings --- Warning control Warning Categories The Warnings Filter Default Warning Filters Temporarily Suppressing Warnings Testing Warnings Updating Code For New Versions of Python Available Functions Available Context Managers contextlib --- Utilities for with-statement contexts Utilities Examples and Recipes Supporting a variable number of context managers Simplifying support for single optional context managers Catching exceptions from __enter__ methods Cleaning up in an __enter__ implementation Replacing any use of try-finally and flag variables Using a context manager as a function decorator abc --- Abstract Base Classes atexit --- Exit handlers atexit Example traceback --- Print or retrieve a stack traceback Traceback Examples __future__ --- Future statement definitions gc --- Garbage Collector interface inspect --- Inspect live objects Types and members Retrieving source code Introspecting callables with the Signature object Classes and functions The interpreter stack Fetching attributes statically Current State of a Generator site --- Site-specific configuration hook fpectl --- Floating point exception control Example Limitations and other considerations distutils --- Building and installing Python modules Custom Python Interpreters code --- Interpreter base classes Interactive Interpreter Objects Interactive Console Objects codeop --- Compile Python code Importing Modules imp --- Access the import internals Examples zipimport --- Import modules from Zip archives zipimporter Objects Examples pkgutil --- Package extension utility modulefinder --- Find modules used by a script Example usage of ModuleFinder runpy --- Locating and executing Python modules importlib -- An implementation of import Introduction Functions importlib.abc -- Abstract base classes related to import importlib.machinery -- Importers and path hooks importlib.util -- Utility code for importers Python Language Services parser --- Access Python parse trees Creating ST Objects Converting ST Objects Queries on ST Objects Exceptions and Error Handling ST Objects Example: Emulation of compile() ast --- Abstract Syntax Trees Node classes Abstract Grammar ast Helpers symtable --- Access to the compiler's symbol tables Generating Symbol Tables Examining Symbol Tables symbol --- Constants used with Python parse trees token --- Constants used with Python parse trees keyword --- Testing for Python keywords tokenize --- Tokenizer for Python source Tokenizing Input Command-Line Usage Examples tabnanny --- Detection of ambiguous indentation pyclbr --- Python class browser support Class Objects Function Objects py_compile --- Compile Python source files compileall --- Byte-compile Python libraries Command-line use Public functions dis --- Disassembler for Python bytecode Python Bytecode Instructions pickletools --- Tools for pickle developers Command line usage Command line options Programmatic Interface Miscellaneous Services formatter --- Generic output formatting The Formatter Interface Formatter Implementations The Writer Interface Writer Implementations MS Windows Specific Services msilib --- Read and write Microsoft Installer files Database Objects View Objects Summary Information Objects Record Objects Errors CAB Objects Directory Objects Features GUI classes Precomputed tables msvcrt -- Useful routines from the MS VC++ runtime File Operations Console I/O Other Functions winreg -- Windows registry access Functions Constants HKEY_* Constants Access Rights 64-bit Specific Value Types Registry Handle Objects winsound --- Sound-playing interface for Windows Unix Specific Services posix --- The most common POSIX system calls Large File Support Notable Module Contents pwd --- The password database spwd --- The shadow password database grp --- The group database crypt --- Function to check Unix passwords Hashing Methods Module Attributes Module Functions Examples termios --- POSIX style tty control Example tty --- Terminal control functions pty --- Pseudo-terminal utilities Example fcntl --- The fcntl and ioctl system calls pipes --- Interface to shell pipelines Template Objects resource --- Resource usage information Resource Limits Resource Usage nis --- Interface to Sun's NIS (Yellow Pages) syslog --- Unix syslog library routines Examples Simple example Undocumented Modules Platform specific modules Extending and Embedding the Python Interpreter Extending Python with C or C++ A Simple Example Intermezzo: Errors and Exceptions Back to the Example The Module's Method Table and Initialization Function Compilation and Linkage Calling Python Functions from C Extracting Parameters in Extension Functions Keyword Parameters for Extension Functions Building Arbitrary Values Reference Counts Reference Counting in Python Ownership Rules Thin Ice NULL Pointers Writing Extensions in C++ Providing a C API for an Extension Module Defining New Types The Basics Adding data and methods to the Basic example Providing finer control over data attributes Supporting cyclic garbage collection Subclassing other types Type Methods Finalization and De-allocation Object Presentation Attribute Management Generic Attribute Management Type-specific Attribute Management Object Comparison Abstract Protocol Support Weak Reference Support More Suggestions Building C and C++ Extensions with distutils Distributing your extension modules Building C and C++ Extensions on Windows A Cookbook Approach Differences Between Unix and Windows Using DLLs in Practice Embedding Python in Another Application Very High Level Embedding Beyond Very High Level Embedding: An overview Pure Embedding Extending Embedded Python Embedding Python in C++ Compiling and Linking under Unix-like systems Python/C API Reference Manual Introduction Include Files Objects, Types and Reference Counts Reference Counts Reference Count Details Types Exceptions Embedding Python Debugging Builds Stable Application Binary Interface The Very High Level Layer Reference Counting Exception Handling Exception Objects Unicode Exception Objects Recursion Control Standard Exceptions Utilities Operating System Utilities System Functions Process Control Importing Modules Data marshalling support Parsing arguments and building values Parsing arguments Strings and buffers Numbers Other objects API Functions Building values String conversion and formatting Reflection Codec registry and support functions Codec lookup API Registry API for Unicode encoding error handlers Abstract Objects Layer Object Protocol Number Protocol Sequence Protocol Mapping Protocol Iterator Protocol Buffer Protocol Buffer structure Buffer request types request-independent fields readonly, format shape, strides, suboffsets contiguity requests compound requests Complex arrays NumPy-style: shape and strides PIL-style: shape, strides and suboffsets Buffer-related functions Old Buffer Protocol Concrete Objects Layer Fundamental Objects Type Objects The None Object Numeric Objects Integer Objects Boolean Objects Floating Point Objects Complex Number Objects Complex Numbers as C Structures Complex Numbers as Python Objects Sequence Objects Bytes Objects Byte Array Objects Type check macros Direct API functions Macros Unicode Objects and Codecs Unicode Objects Unicode Type Unicode Character Properties Creating and accessing Unicode strings Deprecated Py_UNICODE APIs Locale Encoding File System Encoding wchar_t Support UCS4 Support Built-in Codecs Generic Codecs UTF-8 Codecs UTF-32 Codecs UTF-16 Codecs UTF-7 Codecs Unicode-Escape Codecs Raw-Unicode-Escape Codecs Latin-1 Codecs ASCII Codecs Character Map Codecs MBCS codecs for Windows Methods & Slots Methods and Slot Functions Tuple Objects Struct Sequence Objects List Objects Mapping Objects Dictionary Objects Other Objects Set Objects Function Objects Instance Method Objects Method Objects File Objects Module Objects Initializing C modules Iterator Objects Descriptor Objects Slice Objects MemoryView objects Weak Reference Objects Capsules Cell Objects Generator Objects DateTime Objects Code Objects Initialization, Finalization, and Threads Initializing and finalizing the interpreter Process-wide parameters Thread State and the Global Interpreter Lock Releasing the GIL from extension code Non-Python created threads High-level API Low-level API Sub-interpreter support Bugs and caveats Asynchronous Notifications Profiling and Tracing Advanced Debugger Support Memory Management Overview Memory Interface Examples Object Implementation Support Allocating Objects on the Heap Common Object Structures Type Objects Number Object Structures Mapping Object Structures Sequence Object Structures Buffer Object Structures Supporting Cyclic Garbage Collection API and ABI Versioning Distributing Python Modules An Introduction to Distutils Concepts & Terminology A Simple Example General Python terminology Distutils-specific terminology Writing the Setup Script Listing whole packages Listing individual modules Describing extension modules Extension names and packages Extension source files Preprocessor options Library options Other options Relationships between Distributions and Packages Installing Scripts Installing Package Data Installing Additional Files Additional meta-data Debugging the setup script Writing the Setup Configuration File Creating a Source Distribution Specifying the files to distribute Manifest-related options Creating Built Distributions Creating RPM packages Creating Windows Installers Cross-compiling on Windows The Postinstallation script Vista User Access Control (UAC) The Python Package Index (PyPI) Registering Packages Uploading Packages The .pypirc file PyPI package display Examples Pure Python distribution (by module) Pure Python distribution (by package) Single extension module Checking a package Extending Distutils Integrating new commands Adding new distribution types Command Reference Installing modules: the install command family install_data install_scripts Creating a source distribution: the sdist command API Reference distutils.core --- Core Distutils functionality distutils.ccompiler --- CCompiler base class distutils.unixccompiler --- Unix C Compiler distutils.msvccompiler --- Microsoft Compiler distutils.bcppcompiler --- Borland Compiler distutils.cygwincompiler --- Cygwin Compiler distutils.emxccompiler --- OS/2 EMX Compiler distutils.archive_util --- Archiving utilities distutils.dep_util --- Dependency checking distutils.dir_util --- Directory tree operations distutils.file_util --- Single file operations distutils.util --- Miscellaneous other utility functions distutils.dist --- The Distribution class distutils.extension --- The Extension class distutils.debug --- Distutils debug mode distutils.errors --- Distutils exceptions distutils.fancy_getopt --- Wrapper around the standard getopt module distutils.filelist --- The FileList class distutils.log --- Simple PEP 282-style logging distutils.spawn --- Spawn a sub-process distutils.sysconfig --- System configuration information distutils.text_file --- The TextFile class distutils.version --- Version number classes distutils.cmd --- Abstract base class for Distutils commands Creating a new Distutils command distutils.command --- Individual Distutils commands distutils.command.bdist --- Build a binary installer distutils.command.bdist_packager --- Abstract base class for packagers distutils.command.bdist_dumb --- Build a "dumb" installer distutils.command.bdist_msi --- Build a Microsoft Installer binary package distutils.command.bdist_rpm --- Build a binary distribution as a Redhat RPM and SRPM distutils.command.bdist_wininst --- Build a Windows installer distutils.command.sdist --- Build a source distribution distutils.command.build --- Build all files of a package distutils.command.build_clib --- Build any C libraries in a package distutils.command.build_ext --- Build any extensions in a package distutils.command.build_py --- Build the .py/.pyc files of a package distutils.command.build_scripts --- Build the scripts of a package distutils.command.clean --- Clean a package build area distutils.command.config --- Perform package configuration distutils.command.install --- Install a package distutils.command.install_data --- Install data files from a package distutils.command.install_headers --- Install C/C++ header files from a package distutils.command.install_lib --- Install library files from a package distutils.command.install_scripts --- Install script files from a package distutils.command.register --- Register a module with the Python Package Index distutils.command.check --- Check the meta-data of a package Installing Python Modules Introduction Best case: trivial installation The new standard: Distutils Standard Build and Install Platform variations Splitting the job up How building works How installation works Alternate Installation Alternate installation: the user scheme Alternate installation: the home scheme Alternate installation: Unix (the prefix scheme) Alternate installation: Windows (the prefix scheme) Custom Installation Modifying Python's Search Path Distutils Configuration Files Location and names of config files Syntax of config files Building Extensions: Tips and Tricks Tweaking compiler/linker flags Using non-Microsoft compilers on Windows Borland/CodeGear C++ GNU C / Cygwin / MinGW Older Versions of Python and MinGW Python HOWTOs Porting Python 2 Code to Python 3 Choosing a Strategy Universal Bits of Advice Python 3 and 3to2 Python 2 and 2to3 Support Python 2.7 Try to Support Python 2.6 and Newer Only from __future__ import print_function from __future__ import unicode_literals Bytes literals Supporting Python 2.5 and Newer Only from __future__ import absolute_import Mark all Unicode strings with a u prefix Handle Common "Gotchas" from __future__ import division Specify when opening a file as binary Text files Subclass object Deal With the Bytes/String Dichotomy Mark Up Python 2 String Literals Decide what APIs Will Accept Bytes / Unicode Comparison Indexing bytes objects __str__()/__unicode__() Don't Index on Exceptions Don't use __getslice__ & Friends Updating doctests Update map for imbalanced input sequences Eliminate -3 Warnings Run 2to3 Manually During Installation Verify & Test Python 2/3 Compatible Source Follow The Steps for Using 2to3 Use six Capturing the Currently Raised Exception Other Resources Porting Extension Modules to Python 3 Conditional compilation Changes to Object APIs str/unicode Unification long/int Unification Module initialization and state CObject replaced with Capsule Other options Curses Programming with Python What is curses? The Python curses module Starting and ending a curses application Windows and Pads Displaying Text Attributes and Color User Input For More Information Descriptor HowTo Guide Abstract Definition and Introduction Descriptor Protocol Invoking Descriptors Descriptor Example Properties Functions and Methods Static Methods and Class Methods Functional Programming HOWTO Introduction Formal provability Modularity Ease of debugging and testing Composability Iterators Data Types That Support Iterators Generator expressions and list comprehensions Generators Passing values into a generator Built-in functions The itertools module Creating new iterators Calling functions on elements Selecting elements Grouping elements The functools module The operator module Small functions and the lambda expression Revision History and Acknowledgements References General Python-specific Python documentation Logging HOWTO Basic Logging Tutorial When to use logging A simple example Logging to a file Logging from multiple modules Logging variable data Changing the format of displayed messages Displaying the date/time in messages Next Steps Advanced Logging Tutorial Logging Flow Loggers Handlers Formatters Configuring Logging What happens if no configuration is provided Configuring Logging for a Library Logging Levels Custom Levels Useful Handlers Exceptions raised during logging Using arbitrary objects as messages Optimization Logging Cookbook Using logging in multiple modules Multiple handlers and formatters Logging to multiple destinations Configuration server example Dealing with handlers that block Sending and receiving logging events across a network Adding contextual information to your logging output Using LoggerAdapters to impart contextual information Using objects other than dicts to pass contextual information Using Filters to impart contextual information Logging to a single file from multiple processes Using file rotation Use of alternative formatting styles Customising LogRecord Subclassing QueueHandler - a ZeroMQ example Subclassing QueueListener - a ZeroMQ example An example dictionary-based configuration Using a rotator and namer to customise log rotation processing A more elaborate multiprocessing example Inserting a BOM into messages sent to a SysLogHandler Implementing structured logging Regular Expression HOWTO Introduction Simple Patterns Matching Characters Repeating Things Using Regular Expressions Compiling Regular Expressions The Backslash Plague Performing Matches Module-Level Functions Compilation Flags More Pattern Power More Metacharacters Grouping Non-capturing and Named Groups Lookahead Assertions Modifying Strings Splitting Strings Search and Replace Common Problems Use String Methods match() versus search() Greedy versus Non-Greedy Using re.VERBOSE Feedback Socket Programming HOWTO Sockets History Creating a Socket IPC Using a Socket Binary Data Disconnecting When Sockets Die Non-blocking Sockets Sorting HOW TO Sorting Basics Key Functions Operator Module Functions Ascending and Descending Sort Stability and Complex Sorts The Old Way Using Decorate-Sort-Undecorate The Old Way Using the cmp Parameter Odd and Ends Unicode HOWTO Introduction to Unicode History of Character Codes Definitions Encodings References Python's Unicode Support The String Type Converting to Bytes Unicode Literals in Python Source Code Unicode Properties Unicode Regular Expressions References Reading and Writing Unicode Data Unicode filenames Tips for Writing Unicode-aware Programs Converting Between File Encodings Files in an Unknown Encoding References Acknowledgements HOWTO Fetch Internet Resources Using The urllib Package Introduction Fetching URLs Data Headers Handling Exceptions URLError HTTPError Error Codes Wrapping it Up Number 1 Number 2 info and geturl Openers and Handlers Basic Authentication Proxies Sockets and Layers Footnotes HOWTO Use Python in the web The Low-Level View Common Gateway Interface Simple script for testing CGI Setting up CGI on your own server Common problems with CGI scripts mod_python FastCGI and SCGI Setting up FastCGI mod_wsgi Step back: WSGI WSGI Servers Case study: MoinMoin Model-View-Controller Ingredients for Websites Templates Data persistence Frameworks Some notable frameworks Django TurboGears Zope Other notable frameworks Argparse Tutorial Concepts The basics Introducing Positional arguments Introducing Optional arguments Short options Combining Positional and Optional arguments Getting a little more advanced Conflicting options Conclusion An introduction to the ipaddress module Creating Address/Network/Interface objects A Note on IP Versions IP Host Addresses Defining Networks Host Interfaces Inspecting Address/Network/Interface Objects Networks as lists of Addresses Comparisons Using IP Addresses with other modules Getting more detail when instance creation fails Python Frequently Asked Questions General Python FAQ General Information Python in the real world Programming FAQ General Questions Core Language Numbers and strings Performance Sequences (Tuples/Lists) Dictionaries Objects Modules Design and History FAQ Why does Python use indentation for grouping of statements? Why am I getting strange results with simple arithmetic operations? Why are floating-point calculations so inaccurate? Why are Python strings immutable? Why must 'self' be used explicitly in method definitions and calls? Why can't I use an assignment in an expression? Why does Python use methods for some functionality (e.g. list.index()) but functions for other (e.g. len(list))? Why is join() a string method instead of a list or tuple method? How fast are exceptions? Why isn't there a switch or case statement in Python? Can't you emulate threads in the interpreter instead of relying on an OS-specific thread implementation? Why can't lambda expressions contain statements? Can Python be compiled to machine code, C or some other language? How does Python manage memory? Why doesn't CPython use a more traditional garbage collection scheme? Why isn't all memory freed when CPython exits? Why are there separate tuple and list data types? How are lists implemented? How are dictionaries implemented? Why must dictionary keys be immutable? Why doesn't list.sort() return the sorted list? How do you specify and enforce an interface spec in Python? Why are default values shared between objects? Why is there no goto? Why can't raw strings (r-strings) end with a backslash? Why doesn't Python have a "with" statement for attribute assignments? Why are colons required for the if/while/def/class statements? Why does Python allow commas at the end of lists and tuples? Library and Extension FAQ General Library Questions Common tasks Threads Input and Output Network/Internet Programming Databases Mathematics and Numerics Extending/Embedding FAQ Can I create my own functions in C? Can I create my own functions in C++? Writing C is hard; are there any alternatives? How can I execute arbitrary Python statements from C? How can I evaluate an arbitrary Python expression from C? How do I extract C values from a Python object? How do I use Py_BuildValue() to create a tuple of arbitrary length? How do I call an object's method from C? How do I catch the output from PyErr_Print() (or anything that prints to stdout/stderr)? How do I access a module written in Python from C? How do I interface to C++ objects from Python? I added a module using the Setup file and the make fails; why? How do I debug an extension? I want to compile a Python module on my Linux system, but some files are missing. Why? What does "SystemError: _PyImport_FixupExtension: module yourmodule not loaded" mean? How do I tell "incomplete input" from "invalid input"? How do I find undefined g++ symbols __builtin_new or __pure_virtual? Can I create an object class with some methods implemented in C and others in Python (e.g. through inheritance)? Python on Windows FAQ How do I run a Python program under Windows? How do I make Python scripts executable? Why does Python sometimes take so long to start? How do I make an executable from a Python script? Is a *.pyd file the same as a DLL? How can I embed Python into a Windows application? How do I keep editors from inserting tabs into my Python source? How do I check for a keypress without blocking? How do I emulate os.kill() in Windows? How do I extract the downloaded documentation on Windows? Graphic User Interface FAQ General GUI Questions What platform-independent GUI toolkits exist for Python? What platform-specific GUI toolkits exist for Python? Tkinter questions "Why is Python Installed on my Computer?" FAQ What is Python? Why is Python installed on my machine? Can I delete Python? Glossary About these documents Contributors to the Python Documentation Reporting Bugs Documentation bugs Using the Python issue tracker Copyright History and License History of the software Terms and conditions for accessing or otherwise using Python Licenses and Acknowledgements for Incorporated Software Mersenne Twister Sockets Floating point exception control Asynchronous socket services Cookie management Execution tracing UUencode and UUdecode functions XML Remote Procedure Calls test_epoll Select kqueue strtod and dtoa OpenSSL expat libffi zlib Get in touch Submit feedback about this site to: [email protected] © documentation.help. Design: rehmann.co.