The Nebula Device 3: IO Namespace Reference

The Nebula Device 3

previous page next page

IO Namespace Reference

Detailed Description

The Nebula3 IO Subsystem

The Nebula3 IO subsystem is a huge step forward from the Nebula1 and Nebula2 IO systems. The main design goals of the new IO subsystem are:

use more standard mechanisms, like URIs to identify resource locations, and MIME types to identify data formats
a flexible stream model, it shouldn't matter whether data comes from a file, from a memory buffer, an HTTP connection or somewhere else
reading and writing data from and to a stream in different data formats should be more orthogonal, for instance it shouldn't matter if XML-formatted data is read from memory, from a file, from a network connection or from anywhere else
extensibility, new stream and reader/writer classes can be registered with the IO subsystem at runtime
portability without performance compromises, the entire IO subsystem must be able to use platform-specific IO functions under the hood instead of relying to CLib functions like fopen() for portability, which may come with an additional performance or memory overhead compared to the platform specific IO functions

The main concepts of the Nebula3 IO subsystem are:

A central IO::Console object for all text input and output with attachable console handlers. It is guaranteed that all Nebula3 text output goes through the console as the one centralized in/out channel. Specialized console handlers can be used to handle text output in a special way (for instance writing the output to stdout, an ingame console, a log file or a network connection).
Assigns as path aliases. The general functionality is the same as in Nebula1 and Nebula2, or the original AmigaOS assigns which inspired Nebula's assign mechanism. A new feature of Nebula3 assigns is that they can be aliases for complete URI's. For instance, the assign "textures:" could be defined as "http://www.radonlabs.de/textures", so that the shortcut resource path "textures:mytexture.dds" would be resolve to the absolute location "http://www.radonlabs.de/textures/mytexture.dds"
Streams as basic in/out data channel. Streams are the replacement for Nebula2's nFile objects. Streams offer the same basic API with Open()/Close()/Read()/Write(), but may hide completely different transport or storage channels behind their common interface. Examples of stream classes are IO::FileStream, IO::MemoryStream, or Net::HttpStream
Stream readers and writers are attached to streams and generally implement easy-to-use interfaces to read and write different data formats. For instance one could attach an IO::XmlReader to a IO::FileStream to read XML-formatted data from a filesystem file, or attach it to an IO::HttpStream to read XML-formatted data from a HTTP connection.

A good example to show the power of the Nebula3 in/out system is the following code fragment:

    IO::FileServer::Instance()->CopyFile("http://www.radonlabs.de/index.html", "temp:index.html");

This single line of code copies a file from a HTTP server into the current user's temp directory. With only a few lines more you could create a stream object pointing to a HTML file on a HTTP server, attach an XML reader to the stream, and parse the content of the HTML file without any intermediate storage file.

Nebula3 Standard Assigns

Nebula3 initializes the following standard assigns:

home: Points to the application directory, in a German Windows installation, this is usually somewhere under "C:/Programme". Nebula3 applications should treat the home: location as a read only directory so that the user doesn't need administrator rights to run the application.
user: This points to the currently logged in user directory for this Nebula3 application. In a German Windows installation, this is somewhere under "C:/Eigene Dateien/[username]". Nebula3 will automatically create a local directory in the user directory to prevent different applications to overwrite their data. It is generally safe to write data to the user directory. This is the place where configuration and save game data should be written, or any other data which should persist between application invokations.
temp: This assign points to the current user's temp directory. This directory is generally writable. It should not be assumed that data in temp: survives until the next application start.
bin: This points to the directory of the application's executable file. This may or may not be identical with the home: directory. The bin: assign should be treated as read-only.

Custom assigns may be defined at runtime by the application. Often this is used to define abstract path to resources like textures, sound data, and so on. That way the locations of those resources can be easily changes by setting a single assign instead of fixing all the resource paths. A nice side effect of assigns is that a path with assigns is often much shorter then an "absolute" path resulting in a smaller memory footprint.

Nebula3 URIs

Resource locations are generally defined through standard URIs in Nebula3. URIs may consist of the following parts, some of them optional:

a scheme, for instance "http:", "file:", etc... Nebula3 doesn't define any hardcoded schemes, instead, schemes are bound to stream classes by registering them with the IO::StreamServer singleton
an optional user info field, often this is a login name and a password to authenticate with a remote FTP or HTTP host
a hostname, like "www.radonlabs.de"
an optional portname following the hostname
a local path, pointing to a resource on the host
an optional fragment, which often points to a location inside the resource
an optional query part, which often contains arguments for a PHP script or some similar dynamic response mechamism

The class IO::URI is used to pass URIs around and to crack URI strings into its various components. It should be noted however, that an URI object has a bigger memory footprint compared to storing the URI in a simple string. So sometimes it may be better keep URIs around in strings and only use the IO::URI class to split the URI string into its parts.

Here are some examples for URI's:

file:///c:/temp/bla.txt
file://samba/temp/bla.txt
http://www.radonlabs.de/index.html
http://user:[email protected]:8080/index.html#main

By using Nebula3 assigns you can simplify those complex pathnames a lot. To reference a file in the application directory you can write for instance home:bla.txt which would resolve to something like file:///c:/programme/[myapp]/bla.txt.

Nebula3 Streams, Readers and Writers

Streams provide a common interface for storing or transporting raw data. They replace the nFile class of Nebula2 with a much more general approach for storing, retrieving and transporting data. A stream object provides the traditional Open()/Close()/Read()/Write()/Seek() interface. Some stream classes provide memory mapping, so that data can be read or written by direct memory access. Stream objects use an IO::URI object to define their resource location. Usually, one URI scheme maps to a specific stream class. For instance the URI scheme "http:" usually maps to the Net::HttpStream class, while the scheme "file:" maps to the IO::FileStream class. This mapping is implemented by the StreamServer which constructs a matching stream object given an URI. A Nebula3 application is responsible to provide the mapping of URI scheme to stream classes using the StreamServer::Register() method. This is also the way how new stream classes and schemes are registered with Nebula3.

Important stream classes in Nebula3 are for instance:

IO::FileStream: provides access to the host's filesystem
IO::MemoryStream: a dynamic memory buffer with a stream interface
IO::HttpStream: provides a stream interface to files on a HTTP server

To read and write formatted stream data in a more flexible way then Nebula2, stream reader and stream writer classes have been introduced in Nebula3. Stream reader and writer classes provide a comfortable interface which is specialized on a specific data format. Here are some examples of stream readers and writers provided by Nebula3:

IO::BinaryReader/IOBinaryWriter: read and write binary data
IO::TextReader/IOTextWriter: read and write text and character data
IO::XmlReader/IOXmlWriter: read and write XML formatted data
Messaging::MessageReader/MessagingMessageWriter: Message serialization

Here's a simple example how to access a file on a HTTP server with a XmlReader:

    using namespace IO;
    
    Ptr<Stream> stream = StreamServer::Instance()->CreateStream("http://www.radonlabs.de/index.html");
    Ptr<XmlReader> xmlReader = XmlReader::Create();
    xmlReader->SetStream(stream);
    if (xmlReader->Open())
    {
        // parse content here using the XmlReader interface
    }

The Nebula3 File Server

The Nebula3 IO::FileServer class provides a singleton which offers access to the hosts filesystem for global operations like defining assigns, copying, deleting and checking for existance of files and directories, listing directory contents, and so on.

Here's some sample code fragments for some of the more useful FileServer methods:

    using namespace IO;
    using namespace Util;
    
    FileServer* fs = FileServer::Instance();

    // check if a file or directory exists
    bool fileExists = fs->FileExists("home:bla.txt");
    bool dirExists = fs->DirectoryExists("temp:bla/blub");
    
    // resolve a path with assigns into an absolute filesystem
    // path, this is sometimes necessary to interface with 
    // 3rd party libraries which don't understand Nebula3 paths directly
    String absPath = fs->ResolveAssings("user:myapp/savegames");
    
    // create a directory, note that all missing subdirectories will
    // be created as well
    fs->CreateDirectory("user:myapp/savegames");
    
    // copy and delete files
    fs->CopyFile("home:movie.mpg", "temp:movie.mpg");
    fs->DeleteFile("temp:movie.mpg");
    
    // list files in a directory matching a pattern
    Array<String> files = fs->ListFiles("temp:", "*.txt");
    
    // list all subdirectories in temp:
    Array<String> dirs = fs->ListDirectories("temp:", "*");

The Nebula3 Console

[TODO]


Data Structures
class	Assign
class	BinaryReader
class	BinaryWriter
class	Console
class	ConsoleHandler
class	FileStream
class	FileTime
class	IoServer
class	MediaType
class	MemoryStream
class	Stream
class	StreamReader
class	StreamWriter
class	TextReader
class	TextWriter
class	URI
class	XmlReader
class	XmlWriter
class	ZipFileStream
class	ZipArchive
class	ZipDirEntry
class	ZipFileEntry
class	ZipFileSystem
class	IOInterfaceHandlerBase
class	Interface

previous page start next page