Detailed Description

Rationale

One of the basic operations necessary to perform during communication and storage is generating a formatted buffer coalesced from many data items, perhaps with necessary conversions, for example ISO8583 message packing. This module is an extension of existing ISO8583 message-packing libraries, with additions so it can be used for other packing formats, and can be highly customized by the user.

Introduction

Formatting (or packing) a buffer is the generation of a buffer from a collection of (many) data elements possibly spread in many data structures, with added conversion and formatting. The inverse process is called unpacking.

The design of this module is such that the user needs to create and maintain a single table with the list of fields, and how to format each, and this same structure can be used both for packing and unpacking. Some pre-defined functions for this are included with the library (see Field Formatting Functions), but the user is free to create new ones.

See the documentation of formatField_t and format_t for information on how to fill those structures. See below for more information about how each field is used during packing and unpacking.

Packing

On a call to formatPack() the following algorithm is followed for each field:

1. If filter != NULL and filter(fmt, field) == 0, ignore this field
2. Set bodySize = 0;
3. If body != NULL, call it to write the field data;
   • Set bodySize to the return value of body;
4. If prefix != NULL, call it;
   • Note that prefix can access bodySize to know the number of bytes actually written by body, and size for the user-provided size information;
5. If suffix != NULL, call it;
   • Note that suffix can access bodySize to know the number of bytes actually written by body, and size for the user-provided size information;
6. Swap body and prefix so prefix is before body on the output buffer.

• The formatting of the size and padding is independent of the formatting of the actual field data;
• bodySize rarely needs to be filled by the user, it is filled on runtime with the value returned by body;
• Fields that require padding to specific sizes can break this into a body that write the actual data and a prefix and/or suffix that deals with the padding;
• On the call to prefix or suffix the size field indicates the number given by the caller when filling the formatField_t structure, while bodySize is the actual number of bytes written by body;
• Padding is usually done by a suffix that writes size - bodySize bytes;
• prefix can be used to write a size field with the actual number of bytes written, as stored in bodySize;
• A separator can be generated using the suffix function.

Unpacking

On a call to formatUnpack() the following algorithm is followed for each field:

1. If filter != NULL and filter(fmt, field) == 0, ignore this field
2. Set bodySize = size;
3. If prefix != NULL, call it;
   • If prefix deals with the actual field length, it must fill bodySize with the number of bytes that need to be read!
4. If body != NULL, call it to read the field data;
   • body should always read bodySize bytes, not size bytes;
5. If suffix != NULL, call it;
   • Note that suffix can access bodySize to know the actual size of body in bytes;

• Remember that prefix should set the bodySize field, and that body must use bodySize instead of size to know how many bytes to read (if prefix is NULL then bodySize = size);
• If suffix is used for padding or separator, then it may only skip the necessary number of bytes;

Examples

Here we list a sample of common field types and a simple ISO8583-like field list with a few bits.

Fixed-size Fields

Fixed-size fields are fields where the number of bytes written by body is always equal to size as given by the user on the field table. This usually means that those fields include no prefix or suffix, but they may require a sufix function to write a fixed separator.

Example: a fixed-size field with 16 bytes.

Variable-size Fields

Variable-size fields require a size prefix to indicate the actual size of the body. Usually they are string data (as card-holder name) or numerical data without padding.

Example: a variable-size field of a string.

Padded Fields

Padded fields are a combination of fixed and variable-sized fields. They have a size prefix but are padded to fit exactly size bytes.

Example: a string field padded with spaces to 20 bytes.

Filtering

By filling the filter field of the format_t structure, the caller may allow conditional processing of fields. The most common use is during ISO8583 processing, but the system is flexible enough for other uses. During execution the filter function is called for each field, passing a pointer to both the format_t and the specific formatField_t.

A filter implementation may use the fieldMap pointer to store the set of allowed fields. For example, for ISO8583, the fieldMap could be a pointer to the actual ISO8583 bitmap as in the packed buffer.

A common framework is to use a body writer function that both pack/ unpack the bitmap and set the fieldMap field, for example:

Is a function that checks the bit field->index of fmt->fieldMap to choose if a field should or not be part of the packing/unpacking process.

In order for this to work, the user must also define, inside the field list, where the bitmap is stored, and properly associate the value of the fieldMap field. The following function does both:

And can be used as:

A simple body function

This is a simple function that could be used as the body of a field. It copies variable-sized zero-terminated strings:

Note how it return strlen() on packing, and both uses and return bodySize on unpack.

A simple prefix function

Following the example above, this is a prefix function that write the value of bodySize as a two-digit decimal number:

Note how it return the number of bytes read or written on both pack and unpack, and how it uses bodySize on packing and stores to it on unpacking. (This version uses the functions from Conversion routines).

A simple suffix function

Now a function that does padding with spaces: it writes enough spaces to compensate by the different between size and bodySize, if any.

Note how, on unpacking, the function ignores the actual buffer contents and just return how many bytes need to be skipped over.

A (small) complete example

Putting together the pieces above we can create a simple formatter table for a buffer with only variable-sized string fields.

First, define the list of fields and their format:

Create the format_t instance with the fields defined above:

Set the bits on iso_bitmap to identify which fields are part of the buffer. (When unpacking, the buffer is read from the input buffer).

Enumeration Type Documentation

Function Documentation