Basic API walkthrough

This tutorial will give an overview of the standard TIER api.

Streams

The encoding library does encoding by writing to arbitrary lua streams. There are two kinds of streams, input streams and output streams. The library considers any lua object that has one of the interfaces specified below to be an input/output stream. If a lua object has both interfaces it is both an input stream and an output stream. Streams are created outside of the tier module. They can be files, in memory streams, tcp streams or any other kind of stream a user wants to use.

Input stream interface

--Reads count number of bytes from the stream.
stream:read(count)

--Closes the stream preventing further stream operations.
stream:close()

Output stream interface

--Writes data to the stream. Data must be a lua string.
stream:write(data)

--Flushes any buffered data. 
stream:flush()

--Closes the stream preventing further operations.
stream:close()

Stream readers and Stream writers

When reading from a stream or writing to a stream the api makes use of StreamReader and StreamWriter objects. These objects have the functionality required to map between primitive lua values and their binary representation. These objects are needed by mapping and encoder/decoder objects to write data into the TIER format. The reader and writer objects implements the packing of lua values into binary data.

Constructing readers and writers

--Creates a stream reader.
--stream must be an InputStream
tier.reader(stream)

--Creates a stream writer.
--stream must be an OutputStream
tier.writer(stream)

Reader Interface

--Reads one or more lua values from the 
--stream corresponding to the format value entered. 
--Possible format values:
--  "b" - reads a signed 8-bit integer from the stream
--  "h" - reads a signed 16-bit integer from the stream
--  "i" - reads a signed 32-bit integer from the stream
--  "l" - reads a signed 64-bit integer from the stream
--  "B" - reads an unsigned 8-bit integer from the stream
--  "H" - reads an unsigned 16-bit integer from the stream
--  "I" - reads an unsigned 32-bit integer from the stream
--  "L" - reads an unsigned 64-bit integer from the stream
--  "f" - reads a float from the stream
--  "d" - reads a double from the stream
--  "s" - reads a varint-length delimited byte array from the stream
--  "r" - reads a raw byte array from the stream.  
--        Length is passed in arguments to readf
--  "v" - reads a varint zigzaged value from the stream.
--  "V" - reads a varint from the stream
--  "p" - reads a signed number of xx bits from the stream. 
--        Number of bits is passed in arguments
--  "P" - reads an unsigned number of xx bits from the stream. 
--        Number of bits is passed in arguments
--Example:
--Reads a string value and a 32-bit integer value. 
--local name, age = reader:readf("si") 
reader:readf(fmt_string, ...)

--Align the underlying stream to the specified alignment. 
reader:align(alignment)

--Discards any remaining bits that are left in the current
--byte in the stream. Must be done if bit "p" or "P" is used 
--and the number of bits read is not 8bit aligned. 
reader:discardbits()

Writer interface*

--Writes one or more lua values to the underlying stream
--Possible format values:
--  "b" - writes a signed 8-bit integer
--        The argument must be a lua integer number in range
--  "h" - writes a signed 16-bit integer
--        The argument must be a lua integer number n range
--  "i" - writes a signed 32-bit integer
--        The argument must be a lua integer number n range
--  "l" - writes a signed 64-bit integer
--        The argument must be a lua integer number
--  "B" - writes an unsigned 8-bit integer
--        The argument must be a lua integer number in range
--  "H" - writes an unsigned 16-bit integer
--        The argument must be a lua integer number n range
--  "I" - writes an unsigned 32-bit integer
--        The argument must be a lua integer number n range
--  "L" - writes an unsigned 64-bit integer
--        The argument must be a lua integer number
--  "f" - writes a float
--        The argument must be a lua number
--  "d" - writes a double
--        The argument must be a lua number
--  "s" - writes a varint-length delimited byte array from the stream
--        The argument must be a lua string
--  "r" - writes a raw byte array from the stream.
--        The argument must be a lua string  
--  "v" - writes a zigzaged varint value
--        The argument must be a lua integer
--  "V" - writes a varint value
--        The argument must be a lua integer
--  "p" - writes a signed n-bit number to the stream 
--        First argument is number of bits, 
--        Second argument is a lua integer in range
--  "P" - writes an unsigned n-bit
--        First argument is number of bits,
--        Second argument is a lua integer in range
--Example:
--Writes a string value and a 32-bit integer value
--writer:writef("si", name, age)
writer:writef(fmt_string, ...)

--Align the underlying stream to the specified alignment. 
writer:align(alignment)

--When using the "p" or "P" format option this method must
--be called if the number of bits are not aligned to 8 bits.
--If this is not done the stream will be corrupt. 
writer:flushbits()

--Flushes all partially written values and the underlying stream.
writer:flush()

Encoders and Decoders

To encode application data into a TIER stream the library makes uses encoder objects. Similarly to decode a TIER stream to application data a decoder object is used. These objects have the following interfaces.

Encoder Api

--Creates an encoder object. 
--output_stream must be either an OutputStream 
--              or a StreamWriter
tier.encoder(output)

--encodes a value to the TIER format.
--mapping maps the value to the TIER format.
--value is the lua object to be encoded.
--... contains extra arguments that some mappings use. 
tier:encode(mapping, value, ...) 

--Closes the encoding scope. Preventing any 
--more operations on the encoder object. 
encoder:close()

Decoder Api

--Creates a Decoder object.
--input is either an InputStream containing     
--      data encoded in the TIER format or
--      a StreamReader.
tier.decoder(input)

--Decodes a value from the input.
--mapping is used to map a value in the TIER 
--        stream to a Lua value.
-- ... contains extra arguments that some mappings need. 
--returns a lua value 
decoder:decode(mapping, ...)

--Closes the scope of the decoder object. 
--Preventing any further operations on it.
decoder:close() : void

In the TIER format several consecutive encoded values have the possibility to share state with each other (see the OBJECT tag). The encoder and decoder objects exists for the purpose of providing scopes or boundaries between sets of encoded values. What this means is that all messages that share the same encoder value will exist in the same scope. For simple values this scope is not relevant but for messages that utilize object encoding or embedded encoding it is very relevant. More information about this topic is found in Encoding References and Encoding of embedded data

Since this scope concept is not needed for simple values there exists a few convenience methods for encoding simple values.

Convenience Encoding Api

--This methods is roughly equivalent to 
--local encoder = tier.encoder(stream)
--encoder:encode(mapping, value, ...)
--encoder:close()
tier.encode(stream, value, mapping, ...)

--Encodes the value to a string
--returns a string containing the value in TIER format. 
tier.encodestring(value, mapping, ...) 

--This method is roughly equivalent to
--local decoder = tier.decoder(stream, has_metadata)
--local value   = decoder:decode(mapping)
--decoder:close()
--returns the decoded value
tier.decode(stream, mapping, ...)

###TIER Metatypes TIER has the option to encode metatype information inside it's streams. This metatype information is useful for type checking and automatic decoding of data. The implementation represents this typing as metatype objects. These objects are located in the tier.meta module. This module contains functions for creating type object from their binary form and to encode the metatype data in a tier stream from a metatype object. Further it contains some utility functions to do type checking, simplify debugging and more.

Metatype object interface

--The 'tag' field is a number that contains the 
--top level type of the metatype. 
--For example: The meta type meta.list(...) contains 
--the tag LIST as it's primary type.  
metatype.tag

--Aggregate metatypes such as meta.map(...) and meta.object(...) contains 1 .. N 
--subtypes that can be accessed using the indices 1 .. N
metatype[index]

--The length operator returns the number of subtypes that a metatype has. 
#metatype

Metatypes are memoized

For reasons to do with how the TIER binary metatype format works all metatypes are memoized that is the following holds true.

local meta = require"tier.meta"
local list1 = meta.list(meta.varint)
local list2 = meta.list(meta.varint)

assert(list1 == list2) --This is true. 

Currently the TIER metatype format is fixed and users cannot create new custom metatypes. This limitation might be lifted in the future.

Utility functions

--A human readable representation of the metatype
meta.tostring(metatype)

--Checks if the two types are compatible with each other. 
--Some types in the tier format are not the same type but 
--can be encoded interchangeably this method checks if that
--is this case for metatypeA and metatypeB. 
meta.typecheck(metatypeA, metatypeB)

###Mappings To map between application Lua values and values encoded in a TIER stream the library uses the concept of mapping objects. A mapping object can encode a particular kind of Lua object into a TIER stream and decode that kind of Lua value back from the stream. They do this encoding by using writer objects to encode data and reader objects to decode data. The mappings have the following interface.

Mapping Interface

--The 'meta' field contains a tier metatype object.
--For example: The primitive.varint has the meta object meta.varint. 
--This meta object tells the application what kind of data is being
--encoded/decoded in the stream. 
--NOTE: The application does not check if the mapping actually encodes 
--the data that this meta field describes. Thus if new custom mappings
--are created be sure to use the correct metatype. 
mapping.meta

--Encodes a value to the TIER format.
--encoder is an encoder object that is
--        used to do the actual encoding.
--value is the lua value to encode.
mapping:encode(encoder, value)

--Decodes a value from the TIER format.
--decoder is a decoder object that is 
--        used to do the actual decoding.
--returns a lua value
mapping:decode(decoder)

There are in general two kinds of mappings in the API. primitive mappings and aggregate mappings.

Primitive mappings are mappings that map to and from primitive types, for example primitive.int32 maps from a lua number to a 32-bit integer number. All these mappings are located in the tier.primitive submodule and can be used without having to be constructed.

Aggregate mappings are composed of other mappings to make more complex types such as tuples, lists, maps etc. For example to make a list of integers the standard.list is composed with primitive.int32. The aggregate mappings are all found in the tier.standard module.