NiBackRecorder.cpp - sample program

Source file: Click the following link to view the source code file:

• NiBackRecorder\main.cpp

The program saves the most recently generated depth and image data. The length of the recorded period is configurable. The save mechanism works by storing frames in memory in a cyclic buffer. On user request the program dumps the buffer to an ONI file.

The documentation describes the sample program's code from the top of the program file(s) to bottom.

Every OpenNI feature is described the first time it appears in this sample program. Further appearances of the same feature are not described again.

Program Declarations

The following declares the RGB definitions for two output modes: QVGA and VGA. The mode is the OpenNI type XnMapOutputMode struct. For example, QVGAMode comprises a width+height of resolution 320x240 and a frame rate of 30 fps (frames per second).

            XnMapOutputMode QVGAMode = { 320, 240, 30 };
            XnMapOutputMode VGAMode = { 640, 480, 30 };

The following declaration block declares the RecConfiguration struct to contain the recording configuration. The constructor initializes its values.

            struct RecConfiguration
            {
              ...
            }

The RecConfiguration struct's fields and initializations are described in the following table.

Configuration Setting	Description
pDepthMode	Represents the user setting of the map output mode of the DepthGenerator node. Initialized to QVGAMode.
pImageMode	Represents the user setting of the map output mode of the ImageGenerator node. Initialized to QVGAMode.
bRecordDepth	Represents the user setting of whether to record the output of the DepthGenerator node. Initialized to FALSE.
bRecordImage	Represents the user setting of whether to record the output of the ImageGenerator node. Initialized to FALSE.
bMirrorIndicated	Indicates that the user provided the bMirror setting to enable or disable a node's mirroring (assuming the node supports mirroring). Initialized to FALSE (i.e., at initialization time the user has not yet supplied a parameter).
bMirror	Represents the user setting of whether to enable or disable a node's mirroring (assuming the node supports mirroring). Initialized to TRUE.
bRegister	Indicates whether registration between depth and image should be activated through OpenNI's AlternativeViewPoint capability.
bFrameSync	Represents the user setting of whether to synchronize between depth and image. That is, the depth and image frames that arrive from the sensor will always arrive in matching pairs (as indicated by the frames' timestamp fields).
bVerbose	Set to TRUE to turn on the log; FALSE to turn off the log, and no log messages at all are printed.
nDumpTime	Represents the user's time period setting for the buffer size. This is the number of seconds of data that can be saved to the cyclic buffer without overwriting previous saved data.
strDirName	Represents the user setting of the full path name, including the file name, to where to write the output data file.

ConfigureGenerators() function

This function configures the generator nodes. The function takes references to generator parameters, as shown in the function's header below. The OpenNI objects passed by these parameters help produce the OpenNI production graph and they are described separately in the following paragraphs.

            XnStatus ConfigureGenerators(const RecConfiguration& config, xn::Context& context, xn::DepthGenerator& depthGenerator, xn::ImageGenerator& imageGenerator)
            {
             ...
            }

Meaning of the Parameters of the ConfigureGenerators() function

The production graph is a network of objects - called production nodes - that work together to produce data for Natural Interface applications. xn::Generator "Generator" nodes are a particular type of production node, which generate data.

A Context object is a workspace in which the application builds an OpenNI production graph.

                xn::Context& context

A DepthGenerator node generates a depth map. Each map pixel value represents a distance from the sensor.

                xn::DepthGenerator& depthGenerator

An ImageGenerator node generates color image maps of various formats, such as the RGB24 image format.

                xn::ImageGenerator& imageGenerator

The body of the ConfigureGenerators() function is described in the following subsections.

Meaning of the Parameters of the ConfigureGenerators() function

The declaration block at the top of the main program declares an OpenNI status flag for collecting return values from method calls. It initializes it with xn::XN_STATUS_OK, the OpenNI value for valid status. Also declared is an OpenNI xn::EnumerationErrors errorlist for collecting any errors that may occur.

                XnStatus nRetVal = XN_STATUS_OK;
                xn::EnumerationErrors errors;

Configures the DepthGenerator node

Configures the DepthGenerator node, if needed. OpenNI functions in the body of this 'if' branch are described separately.

                if (config.bRecordDepth)
                {
                    ...
                }

In the following statement, the call to xn::Context::CreateAnyProductionTree() enumerates for production nodes of a specific node type, and creates the first production node found of that type. A reference to a DepthGenerator node is returned in depthGenerator.

                nRetVal = context.CreateAnyProductionTree(XN_NODE_TYPE_DEPTH, NULL, depthGenerator, &errors);

See also Understanding the Create() method for more detail.

The following statement sets the map output mode of a DepthGenerator node.

                nRetVal = depthGenerator.SetMapOutputMode(*config.pDepthMode);

The following code turns on the requested capabilities if they are supported. For example, the following code checks if the "DepthGenerator" node supports the MirrorCapability, and if so, it gets the capability object and enables the node's mirroring.

                if (config.bMirrorIndicated && depthGenerator.IsCapabilitySupported(XN_CAPABILITY_MIRROR))
                {
                    depthGenerator.GetMirrorCap().SetMirror(config.bMirror);
                }           

Class CyclicBuffer

The CyclicBuffer class provides the cyclic buffer for this sample program. The program can write frames to the buffer, and then read frames back from the buffer to dump them to output files.

m_pFrames is the data structure actually implementing the cyclic buffer.

At this point the reader may find it useful to already study Declaration Block for the CyclicBuffer Class to learn the meaning of data structure and then return here to continue learning the code of the CyclicBuffer class in order.

Class Constructor

This constructor sets up references to the generator nodes that are parsed as parameters to this constructor, as follows.

                CyclicBuffer(xn::Context& context, xn::DepthGenerator& depthGenerator, xn::ImageGenerator& imageGenerator,
                    const RecConfiguration& config) :
                                m_context(context),
                                m_depthGenerator(depthGenerator),
                                m_imageGenerator(imageGenerator)

Initialize() method

This method initializes the m_pFrames cyclic buffer and also the output directory path. xnOSStrCopy () copies one string from the other, making a local copy of the output path. The buffer size is in number of frames (seconds x 30 FPS).

                void Initialize(XnChar* strDirName, XnUInt32 nSeconds)
                {
                    xnOSStrCopy(m_strDirName, strDirName, XN_FILE_MAX_PATH);
                    m_nBufferSize = nSeconds*30;
                    m_pFrames = XN_NEW_ARR(SingleFrame, m_nBufferSize);
                }

This method is invoked from the main() function main() program function.

Declaring the <code>m_pFrames</code> Cyclic Buffer

In the above Initialize) method, the macro XN_NEW_ARR() allocates an array of frames

where the SingleFrame type is defined as follows.

                struct SingleFrame
                {
                    xn::DepthMetaData depthFrame;
                    xn::ImageMetaData imageFrame;
                };          

Thus each entry in the cyclic buffer is a frame comprising a depth map data frame and an image map data frame.

The CyclicBuffer class's declaration block is described in Declaration Block for the CyclicBuffer Class.

Update() method

This method saves new OpenNI data in the cyclic buffer. This method is called from the main program loop, immediately after the WaitAndUpdate() call. See Sample Code for the work cycle summary.

Function heading:

                void Update(const xn::DepthGenerator& depthGenerator, const xn::ImageGenerator& imageGenerator)
                {
                    ...
                }

The following code block gets the latest available data generated by the DepthGenerator node, saving it to a frame object.

DepthMetaData provides the frame object for the DepthGenerator node. The frame object provides access to a snapshot of a generator node's data frame and all its associated properties.

If requested, this code copies the DepthGenerator frame object to the m_pFrames cyclic buffer.

                if (m_bDepth)
                {
                    xn::DepthMetaData dmd;
                    depthGenerator.GetMetaData(dmd);
                    m_pFrames[m_nNextWrite].depthFrame.CopyFrom(dmd);
                }

As for the DepthGenerator above, the following code block gets the latest available data generated by the ImageGenerator node, saving it to a frame object.

The rest of the code in this function manages the indexes of the cyclic buffer.

Dump() method

This method saves the current state of the cyclic buffer to a file. This method sets up a Recorder node and that makes the program record all data generated from the mock nodes. Recording means saving all the data to a disk file. The Recorder.Record() method has to be called to save each frame.

This method is called from the main program loop as a user menu option.

Function heading:

                XnStatus Dump()
                {
                    ...
                }

Declarations

The following declares mock nodes: MockDepthGenerator for the DepthGenerator node and MockImageGenerator for the ImageGenerator node. These are to simulate the actual nodes when recording or playing data from a recording. A mock node does not contain any logic for generating data. Instead, it allows an outside component (such as an application or a real node implementation) to feed it data and configuration changes.

                    xn::MockDepthGenerator mockDepth;
                    xn::MockImageGenerator mockImage;

The EnumerationErrors and XnStatus declarations at the top of the main program collect and report status and errors from OpenNI operations.

                    xn::EnumerationErrors errors;
                    XnStatus rc;

Declarations

CreateAnyProductionTree() method creates a Recorder node. The m_recorder parameter returns a reference to a Recorder node.

                    rc = m_context.CreateAnyProductionTree(XN_NODE_TYPE_RECORDER, NULL, m_recorder, &errors);

bkrec_class_cyc_setsup_rec_dest

In the following statement, the call to SetDestination() specifies to where the recorder must send its recording. This is a disk file of a particular file type.

                    m_recorder.SetDestination(XN_RECORD_MEDIUM_FILE, strFileName);

bkrec_class_cyc_crt_mock_nodes

If requested, in the following code block, CreateMockNodeBasedOn() creates a mock node based on the DepthGenerator node. This means the properties of the original DepthGenerator object will be copied to the newly created MockDepthGenerator, and it is required to make the mock depth generator work properly.

AddNodeToRecording() adds the mockDepth node to the recording setup, and starts recording data that the node generates.

                    if (m_bDepth)
                    {
                        rc = m_context.CreateMockNodeBasedOn(m_depthGenerator, NULL, mockDepth);
                        rc = m_recorder.AddNodeToRecording(mockDepth, XN_CODEC_16Z_EMB_TABLES);
                    }

If requested, in the following code block, CreateMockNodeBasedOn() creates a mock node based on the ImageGenerator node.

AddNodeToRecording() adds the mockImage node to the recording setup, and starts recording data that the node generates.

                    if (m_bDepth)
                    {
                        rc = m_context.CreateMockNodeBasedOn(m_imageGenerator, NULL, mockImage);
                        rc = m_recorder.AddNodeToRecording(mockImage, XN_CODEC_JPEG);
                    }

bkrec_class_cyc_write_frames

The following loops record all the frames that are in the cyclic buffer. The Record()method has to be called to save each frame.

This block is executed only if the m_nNextWrite index has wrapped around. In this case, the method must record the remainder of the nodes until the end of the cyclic buffer. Then the method must start again from 0 until the most recent node that was written.

                    if (m_nNextWrite < m_nBufferCount) 
                       for (XnUInt32 i = m_nNextWrite; i < m_nBufferSize; ++i)
                         ...
                    
                    for (XnUInt32 i = 0; i < m_nNextWrite; ++i)
                        ...

In both the above loops the MockDepthGenerator.SetData() method copies all the frame data from the cyclic buffer into the two mock nodes.

                    mockDepth.SetData(m_pFrames[i].depthFrame);
                    mockImage.SetData(m_pFrames[i].imageFrame);

On completion of the record loops, the following code block unreferences each of the production nodes below, for each node decreasing its reference count by 1. If the reference count of any of the nodes reaches zero, OpenNI destroys the node.

                    m_recorder.Release();
                    mockImage.Release();
                    mockDepth.Release();

If excution of the Dump() method got this far, it must have been successful.

                    return XN_STATUS_OK;

Declaration Block for the CyclicBuffer Class

Following is the declaration block for the CyclicBuffer Class.

                struct SingleFrame
                {
                    xn::DepthMetaData depthFrame;
                    xn::ImageMetaData imageFrame;
                };

                XnBool m_bDepth, m_bImage;
                SingleFrame* m_pFrames;
                XnUInt32 m_nNextWrite;
                XnUInt32 m_nBufferSize;
                XnUInt32 m_nBufferCount;
                XnChar m_strDirName[XN_FILE_MAX_PATH];

                xn::Context& m_context;
                xn::DepthGenerator& m_depthGenerator;
                xn::ImageGenerator& m_imageGenerator;
                xn::Recorder m_recorder;

The RecConfiguration struct's fields and initializations are described in the following table. Only the data items comprising OpenNI elements are listed.

Item	Description
SingleFrame	This is a single frame of the m_pFrames cyclic buffer. The frame contains a DepthMetaData Frame Objects and a ImageGenerator Frame Objects.
m_pFrames	This is the cyclic buffer data structure itself.
XnChar m_strDirName[XN_FILE_MAX_PATH]	To hold the output file name.
xn::Context& m_context;	Declares a local reference within the CyclicBuffer class to call the CreateAnyProductionTree() method. See ConfigureGenerators() function.
xn::DepthGenerator& m_depthGenerator;	Declares a reference within the CyclicBuffer object to assign a DepthGenerator frame object to the CyclicBuffer. See ConfigureGenerators() function.
xn::ImageGenerator& m_imageGenerator;	Declares a reference within the CyclicBuffer object to assign an ImageGenerator frame object to the CyclicBuffer. See ConfigureGenerators() function.
xn::Recorder m_recorder;	The Recorder node for saving the frame data generated by the production graph. See ConfigureGenerators() function.

main() function

The OpenNi objects. For description see ConfigureGenerators() function.

            xn::Context context;
            xn::DepthGenerator depthGenerator;
            xn::ImageGenerator imageGenerator;

To count missed frames: Not OpenNI specific.

RecConfiguration config; See Program Declarations.

Parse the command line arguments: Not OpenNI specific.

Turn on log:

Initialize OpenNI:

            nRetVal = context.Init();

Configure the generator nodes.

            nRetVal = ConfigureGenerators(config, context, depthGenerator, imageGenerator);

Ensures all created Generator Node generators are generating data.

            nRetVal = context.StartGeneratingAll();

Create and initialize the cyclic buffer. See Class Constructor and Initialize() method.

            CyclicBuffer cyclicBuffer(context, depthGenerator, imageGenerator, config);
            cyclicBuffer.Initialize(config.strDirName, config.nDumpTime);

main() function

Here is the main loop.

                while (1)
                {
                    ...
                }

The following call to a 'Wait And Update All' method updates the data available for output. The WaitAndUpdateAll() method updates all generator nodes in the context's Production Graph to the latest available data, first waiting for all nodes to have new data available. (In this sample the Production Graph is the DepthGenerator and ImageGenerator.) The application then reads this data through the frame object.

                context.WaitAndUpdateAll();