14.5 logging -- Logging facility for Python

Python 2.5

previous page next page

14.5 `logging` -- Logging facility for Python

New in version 2.3. This module defines functions and classes which implement a flexible error logging system for applications.

Logging is performed by calling methods on instances of the Logger class (hereafter called loggers). Each instance has a name, and they are conceptually arranged in a name space hierarchy using dots (periods) as separators. For example, a logger named "scan" is the parent of loggers "scan.text", "scan.html" and "scan.pdf". Logger names can be anything you want, and indicate the area of an application in which a logged message originates.

Logged messages also have levels of importance associated with them. The default levels provided are DEBUG, INFO, WARNING, ERROR and CRITICAL. As a convenience, you indicate the importance of a logged message by calling an appropriate method of Logger. The methods are debug(), info(), warning(), error() and critical(), which mirror the default levels. You are not constrained to use these levels: you can specify your own and use a more general Logger method, log(), which takes an explicit level argument.

The numeric values of logging levels are given in the following table. These are primarily of interest if you want to define your own levels, and need them to have specific values relative to the predefined levels. If you define a level with the same numeric value, it overwrites the predefined value; the predefined name is lost.

Level	Numeric value
`CRITICAL`	50
`ERROR`	40
`WARNING`	30
`INFO`	20
`DEBUG`	10
`NOTSET`	0

Levels can also be associated with loggers, being set either by the developer or through loading a saved logging configuration. When a logging method is called on a logger, the logger compares its own level with the level associated with the method call. If the logger's level is higher than the method call's, no logging message is actually generated. This is the basic mechanism controlling the verbosity of logging output.

Logging messages are encoded as instances of the LogRecord class. When a logger decides to actually log an event, a LogRecord instance is created from the logging message.

Logging messages are subjected to a dispatch mechanism through the use of handlers, which are instances of subclasses of the Handler class. Handlers are responsible for ensuring that a logged message (in the form of a LogRecord) ends up in a particular location (or set of locations) which is useful for the target audience for that message (such as end users, support desk staff, system administrators, developers). Handlers are passed LogRecord instances intended for particular destinations. Each logger can have zero, one or more handlers associated with it (via the addHandler() method of Logger). In addition to any handlers directly associated with a logger, all handlers associated with all ancestors of the logger are called to dispatch the message.

Just as for loggers, handlers can have levels associated with them. A handler's level acts as a filter in the same way as a logger's level does. If a handler decides to actually dispatch an event, the emit() method is used to send the message to its destination. Most user-defined subclasses of Handler will need to override this emit().

In addition to the base Handler class, many useful subclasses are provided:

StreamHandler instances send error messages to streams (file-like objects).
FileHandler instances send error messages to disk files.
BaseRotatingHandler is the base class for handlers that rotate log files at a certain point. It is not meant to be instantiated directly. Instead, use RotatingFileHandler or TimedRotatingFileHandler.
RotatingFileHandler instances send error messages to disk files, with support for maximum log file sizes and log file rotation.
TimedRotatingFileHandler instances send error messages to disk files rotating the log file at certain timed intervals.
SocketHandler instances send error messages to TCP/IP sockets.
DatagramHandler instances send error messages to UDP sockets.
SMTPHandler instances send error messages to a designated email address.
SysLogHandler instances send error messages to a Unix syslog daemon, possibly on a remote machine.
NTEventLogHandler instances send error messages to a Windows NT/2000/XP event log.
MemoryHandler instances send error messages to a buffer in memory, which is flushed whenever specific criteria are met.
HTTPHandler instances send error messages to an HTTP server using either "GET" or "POST" semantics.

The StreamHandler and FileHandler classes are defined in the core logging package. The other handlers are defined in a sub- module, logging.handlers. (There is also another sub-module, logging.config, for configuration functionality.)

Logged messages are formatted for presentation through instances of the Formatter class. They are initialized with a format string suitable for use with the % operator and a dictionary.

For formatting multiple messages in a batch, instances of BufferingFormatter can be used. In addition to the format string (which is applied to each message in the batch), there is provision for header and trailer format strings.

When filtering based on logger level and/or handler level is not enough, instances of Filter can be added to both Logger and Handler instances (through their addFilter() method). Before deciding to process a message further, both loggers and handlers consult all their filters for permission. If any filter returns a false value, the message is not processed further.

The basic Filter functionality allows filtering by specific logger name. If this feature is used, messages sent to the named logger and its children are allowed through the filter, and all others dropped.

In addition to the classes described above, there are a number of module- level functions.

getLogger( [name]): Return a logger with the specified name or, if no name is specified, return a logger which is the root logger of the hierarchy. If specified, the name is typically a dot-separated hierarchical name like "a", "a.b" or "a.b.c.d". Choice of these names is entirely up to the developer who is using logging.
All calls to this function with a given name return the same logger instance. This means that logger instances never need to be passed between different parts of an application.

getLoggerClass( )

Return either the standard Logger class, or the last class passed to setLoggerClass(). This function may be called from within a new class definition, to ensure that installing a customised Logger class will not undo customisations already applied by other code. For example:

 class MyLogger(logging.getLoggerClass()):
     # ... override behaviour here

debug( msg[, *args[, **kwargs]])

Logs a message with level DEBUG on the root logger. The msg is the message format string, and the args are the arguments which are merged into msg using the string formatting operator. (Note that this means that you can use keywords in the format string, together with a single dictionary argument.)

There are two keyword arguments in kwargs which are inspected: exc_info which, if it does not evaluate as false, causes exception information to be added to the logging message. If an exception tuple (in the format returned by sys.exc_info()) is provided, it is used; otherwise, sys.exc_info() is called to get the exception information.

The other optional keyword argument is extra which can be used to pass a dictionary which is used to populate the __dict__ of the LogRecord created for the logging event with user-defined attributes. These custom attributes can then be used as you like. For example, they could be incorporated into logged messages. For example:

 FORMAT = "%(asctime)-15s %(clientip)s %(user)-8s %(message)s"
 logging.basicConfig(format=FORMAT)
 dict = { 'clientip' : '192.168.0.1', 'user' : 'fbloggs' }
 logging.warning("Protocol problem: %s", "connection reset", extra=d)

would print something like

2006-02-08 22:20:02,165 192.168.0.1 fbloggs  Protocol problem: connection reset

The keys in the dictionary passed in extra should not clash with the keys used by the logging system. (See the Formatter documentation for more information on which keys are used by the logging system.)

If you choose to use these attributes in logged messages, you need to exercise some care. In the above example, for instance, the Formatter has been set up with a format string which expects 'clientip' and 'user' in the attribute dictionary of the LogRecord. If these are missing, the message will not be logged because a string formatting exception will occur. So in this case, you always need to pass the extra dictionary with these keys.

While this might be annoying, this feature is intended for use in specialized circumstances, such as multi-threaded servers where the same code executes in many contexts, and interesting conditions which arise are dependent on this context (such as remote client IP address and authenticated user name, in the above example). In such circumstances, it is likely that specialized Formatters would be used with particular Handlers.

Changed in version 2.5: extra was added.

info( msg[, *args[, **kwargs]]): Logs a message with level INFO on the root logger. The arguments are interpreted as for debug().

warning( msg[, *args[, **kwargs]]): Logs a message with level WARNING on the root logger. The arguments are interpreted as for debug().

error( msg[, *args[, **kwargs]]): Logs a message with level ERROR on the root logger. The arguments are interpreted as for debug().

critical( msg[, *args[, **kwargs]]): Logs a message with level CRITICAL on the root logger. The arguments are interpreted as for debug().

exception( msg[, *args]): Logs a message with level ERROR on the root logger. The arguments are interpreted as for debug(). Exception info is added to the logging message. This function should only be called from an exception handler.

log( level, msg[, *args[, **kwargs]]): Logs a message with level level on the root logger. The other arguments are interpreted as for debug().

disable( lvl): Provides an overriding level lvl for all loggers which takes precedence over the logger's own level. When the need arises to temporarily throttle logging output down across the whole application, this function can be useful.

addLevelName( lvl, levelName): Associates level lvl with text levelName in an internal dictionary, which is used to map numeric levels to a textual representation, for example when a Formatter formats a message. This function can also be used to define your own levels. The only constraints are that all levels used must be registered using this function, levels should be positive integers and they should increase in increasing order of severity.

getLevelName( lvl): Returns the textual representation of logging level lvl. If the level is one of the predefined levels CRITICAL, ERROR, WARNING, INFO or DEBUG then you get the corresponding string. If you have associated levels with names using addLevelName() then the name you have associated with lvl is returned. If a numeric value corresponding to one of the defined levels is passed in, the corresponding string representation is returned. Otherwise, the string "Level %s" % lvl is returned.

makeLogRecord( attrdict): Creates and returns a new LogRecord instance whose attributes are defined by attrdict. This function is useful for taking a pickled LogRecord attribute dictionary, sent over a socket, and reconstituting it as a LogRecord instance at the receiving end.

basicConfig( [**kwargs])

Does basic configuration for the logging system by creating a StreamHandler with a default Formatter and adding it to the root logger. The functions debug(), info(), warning(), error() and critical() will call basicConfig() automatically if no handlers are defined for the root logger.

Changed in version 2.4: Formerly, basicConfig did not take any keyword arguments.

The following keyword arguments are supported.

Format	Description
`filename`	Specifies that a FileHandler be created, using the specified filename, rather than a StreamHandler.
`filemode`	Specifies the mode to open the file, if filename is specified (if filemode is unspecified, it defaults to 'a').
`format`	Use the specified format string for the handler.
`datefmt`	Use the specified date/time format.
`level`	Set the root logger level to the specified level.
`stream`	Use the specified stream to initialize the StreamHandler. Note that this argument is incompatible with 'filename' - if both are present, 'stream' is ignored.

shutdown( ): Informs the logging system to perform an orderly shutdown by flushing and closing all handlers.

setLoggerClass( klass): Tells the logging system to use the class klass when instantiating a logger. The class should define __init__() such that only a name argument is required, and the __init__() should call Logger.__init__(). This function is typically called before any loggers are instantiated by applications which need to use custom logger behavior.