Protocol

This page refers to the protocol used on the initial version of Sirikata and is now quite outdated.

High Level Interaction Overview

In this document we use the word Client interchangeably with the phrase "Object Host" when referring to a 3rd party connecting to a space server to become part of that space. Traditionally Clients host a single object: the camera, which maintains a perspective query of the surrounding environment, but this is left up to the client implementation and the policies in the space: you may find clients hosting avatars, bots, or other objects in a given world.

Object hosts simulate individual objects and facilitate these objects to each connect to one or more spaces. Once an object connects to a space, it may communicate with that space to setup standing queries for nearby or important objects, advertise its position or send messages to objects for which it knows the identifier.

A typical client behavior involves the following steps

• Connect camera object to space server
• Upon connection, register for a new object Id with desired position
• Setup a proximity query for nearby objects that occupy N pixel area on its screen

At this point the camera object will be ready to receive standing query results of nearby objects. As these come in, the client will likely wish to act upon them.

As a proximity request is returned the client is likely to perform the following steps

• Examine the proximity response and understand any bundled data (which may include current location, mesh data, type, etc)
• Construct a message asking that object for remaining data including
  • Mesh URI
  • Position
  • Light information
  • ...

Once the Data is retrieved, the object may be displayed on the client. In the future the client may setup an individual subscription for location updates and other properties of interest with the relevant subscription service. The subscription service is optional and the client must not rely on such a service being present. In the absence of such a service, asking the location service for location updates at a rate proportional to the distance^2 of the object is sufficient.

Clients may wish to interact with objects in the scene (asking them to move, changing their properties, etc). This is accomplished through sending a message to that object on a number of ports. The standard message goes to the object on port 0 and invokes a scripting language function on that object by that name with the provided binary string argument.

Low Level Network Protocols

Sirikata is planned to support 2 low level protocols

• TCP under TCPSST
• UDP under ENET

All components of the system should be able to speak both protocols so the best may be selected for the given circumstance. (Often times ISPs will packet shape UDP and users will get better performance from TCP, but ideally UDP has less overhead than TCP and allows the application to drop outdated packets)

TCPSST

TCPSST is an effort to build a structured streams-like abstraction ontop of TCP by using a handful of individual TCP streams over which to multiplex a number of independent SubStreams. Clients may choose to connect to servers with one or more TCP streams using a handshake that pairs them up appropriately. The default is 3 TCP streams per TCPSST connection. This setup prevents head of line blocking and allows unordered messages to be sent on the least congested stream.

Handshake

every stream begins with the 6 characters SSTTCP then 2 characters delineating in human readable how many streams will be associated with this connection (up to 99) and then a UUID for this connection. This data will be used by the server to pair up the connections and treat them as a single overarching TCPSST stream. The server then sends back a similar handshake header with its own unique UUID to acknowledge the connection pairing.

Framing

After the handshake is complete, data may be sent across the bidirectional sockets as long as each packet of data is preceeded by a packet size (including all following data including the substream identifier) and a substream identifier

Packet size format

The packet size should be a Base 128 VarInt as defined in the protocol buffers wire format. http://code.google.com/apis/protocolbuffers/docs/encoding.html

Substream identifier format

The packet size should be a Base 128 VarInt as defined in the protocol buffers wire format. http://code.google.com/apis/protocolbuffers/docs/encoding.html Substream 0 is reserved for protocol messages.

Application messages

Application messages consist of a positive number of bytes followed encased in the Framing information above. By convention, all messages from a certain stream identifier are sent along a single chosen TCP socket unless the application code specifically marks the unordered option. New streams may be allocated simply by using an unused stream identifier. New streams allocated by the client who initiated the connection should be odd, and new streams allocated by the server who listened for the connection should be even.

Protocol messages

Protocol messages consist of the standard PacketSize followed by the byte 0 which happens to be the VarInt representation for streamid 0 and one of two types of protocol level traffic currently supported

• Closing the stream: byte with the value 0x1 followed by the VarInt streamid to close
  • Close should be sent on all TCPStreams that make up the overarching connection to avoid any unordered packets being delivered late.
• Acking the closing of a stream: byte with the value 0x2 followed by the VarInt streamid
  • An ack should be sent for each close command. Once the ack is received, the streamid may be reused.

Sample Implementation

Sirikata provides a sample C++ implementation of the TCPSST protocol in libcore/plugins/tcpsst This implementation is built atop boost::asio and for the most part uses lock free datastructures or data structures that may be replaced with lock free implementations, so it should be scalable and thread safe.

ENET

The enet module is not yet written, but enet provides a multichannel abstraction over a socket with the additional possibility of a lossy option and portions of a drop-oldest facility. A similar protocol will likely be used with enet, but this is still in the planning stages.

Application Message Serialization

Routing header

The beginning of an application message should start with a header for message routing. The header consists of a set of message fields with id's. These message fields have ID's between 1 and 6 (inclusive) and 1536 and 1791, stored in numerical order.

The message fields are individually serialized as protocol buffers fields http://code.google.com/apis/protocolbuffers/docs/encoding.html

The currently defined fields are

• 1) source object, stored as a variable length array of up to 16 bytes,

zero padded

• 2) destination object, stored as a variable length array of up to 16

bytes, zero padded

• 3) source port, stored as a base 128 variable length up to 32 bit integer
• 4) destination port, stored as a base 128 variable length up to 32 bit integer
• 1536) [extension] source space, stored as a variable length array of up to 16 bytes
• 1537) [extension] destination space, stored as a variable length array of up to 16 bytes

RPC header [optional]

The rpc header follows the routing header and defines fields in the range from 7 to 8 and from 1792 to 2560

Currently defined fields are

• 7) message id (uint64)
• 8) message response id (uint64) indicating one reply to a given message
• 1792) Return Status (enum)

Once a field outside the Routing and RPC headers is encountered, the header parsing is halted and the rest of the message is considered an user level message.

Sending a message id gives the other side the chance to reply in turn with a matching message response id. More than one reply may be matched with a particular message id (as in the case of a standing query, or a persistence request of which some values were in a cache and others needed fetching)

User Message

The user portion of the message is processed as a PBJ message. By default the message is a protocol buffer encoded message.

If the type of the first byte encountered is a Protocol buffer [start group] type, then that byte switches PBJ into one of its advanced modes. These allow it to change the underlying serialization format to thrift or MXP or other formats. Switching to advanced modes is currently unimplemented and considered an extension. PBJ will never support Start Group nor End Group since publicly available tools do not produce them.

Message ports and formats

Each object can have any number of ports. Currently port 0 (scripting port), Persistence, Registration, Loc, Geom/Proximity, and Physics use the RPC format. Persistence uses its own message format that allows for partial responses and Time Sync uses a simple packet format.

RPC Format (MessageBody)

Most messages are in a simple RPC format, which is a list of message names and serialized arguments.

• 9) message_names: Empty if this is a response message. If fewer elements than message_arguments, assume the rest are identical.
• 10) message_arguments: Length must be greater than or equal to the number of message_names. Each is serialized using a predefined format for a given RPC name.

Persistence

ReadWriteSet

message ReadWriteSet {
 repeated StorageElement reads=9;
 repeated StorageElement writes=10;
 flags64 ReadWriteSetOptions {
    RETURN_READ_NAMES=1;
 }
 optional ReadWriteSetOptions options=14;
}

• 9) reads
• 10) writes

For reading or writing individual values, messages must only either send reads or writes in a given message. If a transaction is desired, you must use the MiniTransaction format which isn't really used at all yet.

StorageElement

message StorageElement {
   reserve 1 to 8;//in case we ever need to forward these around a bit
   ///the name of the specific broadcast to listen to
   optional uuid object_uuid=9;
   optional uint64 field_id=10;
   optional string field_name=11;
   optional bytes data=12;
   optional int32 index=13;
   enum ReturnStatus {
       KEY_MISSING = 4;
       INTERNAL_ERROR = 6;
   }
   optional ReturnStatus return_status=15;
   reserve 1536 to 2560;
   reserve 229376 to 294912;
}

Upon reading, set the field_name and object_uuid. Currently only field_id of 0 is supported, however this can possibly be used to implement arrays if desired. The index field will be used for replies that get split up (which is currently allowed)

Responses will have index set (unless the element is sequential), or else have field_data or return_status set (if the field is missing). field_name and object_uuid will be returned only if you request it (and given the nature of the database service, you pretty much have to keep track of what you ask for anyway, so this feature is not very useful).

Time Sync

Time sync allows a client to send a packet with the client_time filled in and a number of reply options set.

message TimeSync {
   reserve 1 to 8;
   reserve 1536 to 2560;
   reserve 229376 to 294912;
   optional time client_time=9;
   optional time server_time=10;
   optional uint64 sync_round=11;
   flags32 ReturnOptions {
        REPLY_RELIABLE=1;
        REPLY_ORDERED=2;
   }
   optional ReturnOptions return_options=14;
   optional time round_trip=2561;
}

The server replies in turn with the server_time filled in and other fields left unchanged

Connection process

After connecting with SST/TCP, objects are created by initiating a new substream and sending a "NewObj" message to Object NULL, Port REGISTRATION as described in the protocol with some randomly generated UUID. Then, you must wait for a RetObj response that acknowledges your object and returns some other ObjectReference UUID that contains your own self reference.

At this point, you are in the system, and you should be prepared to have other objects ask you for your properties or Location at any time.

In order to get a list of other objects in the system, you must send a "NewProxQuery" message to Object NULL, Port GEOM, also containing your location. Currently, unless you intend to test the proximity service, I would recommend putting a large radius in the message, so all in-world objects are considered in range.

At this point, you should receive a large number of "ProxCall" messages with ENTERED_PROXIMITY, and some UUID. In order to find out what that object is, send it "ReadWriteSet" messages to its PERSISTENCE port with some fields set as 'reads', as well as a "LocRequest" message to the default port 0 (both of these should have the header "id" field set and expect a response). The response for "LocRequest" is of type ObjLoc

LocRequest must currently be sent manually. Soon there should be a LOC service available that updates position.

Other messages you can handle include "SetLoc" to port 0 and a readWriteSet PERSISTENCE message with writes, which should update properties on the object if you trust it. Properties beginning with an "_" are currently considered private, but it is ultimately up to an object how it wishes to enforce these rules. Also, be prepared to recieve EXITED_PROXIMITY "ProxCall" messages if an object logs out.

Step by step connection

Registration process: The client sends the space information about its camera object to the space registration service

MESSAGE "Camera Object" to "Space Registration Service"

message Header {
   optional uint32 source_port <- 0
   optional uuid destination_object -> SpaceServiceID();
   optional uint32 destination_port -> Services::REGISTRATION;
}
message MessageBody {
 string message_names[0] -> "NewObj"
 bytes message_body[0] -> message NewObj {
    uuid object_uuid_evidence -> camera internal uuid
    ObjLoc requested_object_loc -> message ObjLoc {
       time timestamp -> now
       vector3d position -> (735, 3047, 966)
       quaternion orientation -> .378, (0, -.925, 0)
       vector3f velocity -> (0, 0, 0)
       vector3f rotational_axis -> (0, 1, 0)
       float angular_speed -> 0
    }
    boundingsphere3f bounding_sphere <- (0,0,0):0
 }
}

The Registration service then replies with a ObjectReference in that space to which other objects may address messages

MESSAGE "Space Registration Service" to "Camera Object"

message Header {
   optional uuid source_object -> SpaceServiceID();
   optional uint32 source_port -> Services::REGISTRATION;
   optional uint32 destination_port <- 0
}
message MessageBody {
 string message_names[0] <- "RetObj"
 bytes message_body[0] <- message RetObj {
    uuid object_reference <- 3d9dac5a-65b5-d333-c21d-970392bcf358
    ObjLoc requested_object_loc <- message ObjLoc {
       time timestamp <- now
       vector3d position <- (735, 3047, 966)
       quaternion orientation <- .378, (0, -.925, 0)
       vector3f velocity <- (0, 0, 0)
       vector3f rotational_axis <- (0, 1, 0)
       float angular_speed <- 0
    }
    boundingsphere3f bounding_sphere <- (0,0,0):0
 }
}

Most camera objects wish to register for objects at least N pixels big. This registration request nabs all objects 10x10 pixels or more on a 1024x768 display

MESSAGE "Camera Object" to "Space Geom Service"

message Header {
   optional uint32 source_port <- 0
   optional uuid destination_object -> SpaceServiceID();
   optional uint32 destination_port -> Services::GEOM;
}
message MessageBody {
 string message_names[0] -> "NewProxQuery"
 string message_body[0] -> message NewProxQuery {
    uint32 query_id -> 0
    angle min_solid_angle -> 5.0e-5
 }
}

The space then replies with a list of objects in range

MESSAGE "Space Geom Service" to "Camera Object"

message Header {
   optional uint32 destination_port <- 0
   optional uuid source_object -> SpaceServiceID();
   optional uint32 source_port -> Services::GEOM;
}
message MessageBody {
 string message_names[0] <- "ProxCall"
 bytes message_body[0] <- message ProxCall {
    uint32 query_id <- 0
    uuid proximate_object <- 3277816b-a486-b9f2-4e07-62f39fb93413
    ProximityEvent proximity_event <- ENTERED_PROXIMITY
 }
 string message_body[1] <- message ProxCall {
    uint32 query_id <- 0
    uuid proximate_object <- d267954d-9215-8bde-935e-470ea13a7781
    ProximityEvent proximity_event <- ENTERED_PROXIMITY
 }
 string message_body[2] <- message ProxCall {
    uint32 query_id <- 0
    uuid proximate_object <- 3d9dac5a-65b5-d333-c21d-970392bcf358
    ProximityEvent proximity_event <- ENTERED_PROXIMITY
 }
}

Now the Camera wishes to obtain location information about these objects

MESSAGE "Camera Object" to 3277816b-a486-b9f2-4e07-62f39fb93413

message Header {
   optional uint32 source_port <- 0
   optional uuid destination_object -> 3277816b-a486-b9f2-4e07-62f39fb93413;
   optional uint32 destination_port -> 0
   
   optional uint64 id -> 1
}
message MessageBody {
 string message_names[0] -> "LocRequest"
 string message_body[0] -> message LocRequest {
 }
}

MESSAGE "Camera Object" to d267954d-9215-8bde-935e-470ea13a7781

message Header {
   optional uint32 source_port <- 0
   optional uuid destination_object -> d267954d-9215-8bde-935e-470ea13a7781
   optional uint32 destination_port -> 0
   
   optional uint64 id -> 2
}
message MessageBody {
 string message_names[0] -> "LocRequest"
 bytes message_body[0] -> message LocRequest {
 }
}

The first object replies with its location information

NOTE: eventually the Loc service should be asked for location information if an authoritative reply is desired

MESSAGE d267954d-9215-8bde-935e-470ea13a7781 to "Camera Object"

message Header {
   optional uint32 destination_port <- 0
   optional uuid source_object -> d267954d-9215-8bde-935e-470ea13a7781
   optional uint32 source_port -> 0
   
   optional uint64 reply_id <- 1
}
message MessageBody {
 string message_names[0] <- "ObjLoc"
 bytes message_body[0] <- message ObjLoc {
   position <- (744.730, 3044.04, 975.9)
   orientation <- 1 (0,0,0)
   velocity <- (0,0,0)
   angular_axis <- (0,1,0)
   angular_speed <- 0
 }
}

The object should also be asked about its appearance and scene graph parent

MESSAGE "Camera Object" to d267954d-9215-8bde-935e-470ea13a7781

message Header {
   optional uint32 source_port <- 0
   optional uuid destination_object -> d267954d-9215-8bde-935e-470ea13a7781
   optional uint32 destination_port -> PERSISTENCE
   
   optional uint64 id -> 3
}
message ReadWriteSet {
 Reads reads[0] -> message Reads {
    string field_name->"MeshURI"
 }
 Reads reads[1] -> message Reads {
    string field_name->"MeshScale"
 }
 Reads reads[2] -> message Reads {
    string field_name->"Name"
 }
 Reads reads[3] -> message Reads {
    string field_name->"PhysicalParameters"
 }
 Reads reads[4] -> message Reads {
    string field_name->"LightInfo"
 }
 Reads reads[5] -> message Reads {
    string field_name->"IsCamera"
 }
 Reads reads[6] -> message Reads {
    string field_name->"Parent"
 }
}

The object then replies with its mesh URI and identity or nulls for other values. This MeshURI may now be streamed in.

MESSAGE d267954d-9215-8bde-935e-470ea13a7781 to "Camera Object"

message Header {
   optional uint32 destination_port -> 0
   optional uuid source_object <- d267954d-9215-8bde-935e-470ea13a7781
   optional uint32 source_port <- PERSISTENCE
   
   optional uint64 reply_id <- 3
}
message ReadWriteSet {
 Reads reads[0] <- message Reads {
    string field_name->"MeshURI"
    bytes data <- message StringProperty {
      string value<- "mhash://cplatz@/arch.mesh"
    }
 }
 Reads reads[1] <- message Reads {
    string field_name <- "MeshScale"
    bytes data <- message VectorProperty {
      vector3f value<- (1, 1, 1)
    }
 }
 Reads reads[2] -> message Reads {
    string field_name->"Name"
    bytes data <- message StringProperty {
      string value<- "Archway"
    }
 }
 Reads reads[3] <- message Reads {
    string field_name<-"PhysicalParameters")
 }
 Reads reads[4] <- message Reads {
    string field_name<-"LightInfo"
 }
 Reads reads[5] -> message Reads {
    string field_name<-"IsCamera"
 }
 Reads reads[6] -> message Reads {
    string field_name<-"Parent"
 }
}

The second object now replies as well, but the details of this conversation are left as an exercise to the reader

MESSAGE 3277816b-a486-b9f2-4e07-62f39fb93413 to "Camera Object"

message Header {
   optional uint32 destination_port <- 0
   optional uuid source_object -> 3277816b-a486-b9f2-4e07-62f39fb93413
   optional uint32 source_port -> 0
   
   optional uint64 reply_id <- 2
}
message MessageBody {
 string message_names[0] <- "ObjLoc"
 bytes message_body[0] <- message ObjLoc {
   position <- (748.730, 3046.04, 973.2)
   orientation <- 1 (0,0,0)
   velocity <- (0,0,0)
   angular_axis <- (0,1,0)
   angular_speed <- 0
 }
}

...

...

...

Updating Your position

the camera may wish to move around and inspect other regions of the world. The camera must let the space know about these movements so the space may deliver new proximity information for new regions

MESSAGE "Camera Object" to "Space Location Service"

message Header {
   optional uint32 source_port <- 0
   optional uuid destination_object -> SpaceServiceID();
   optional uint32 destination_port -> Services::LOC;
}
message MessageBody {
 string message_names[0] -> "ObjLoc"
 bytes message_body[0] -> message ObjLoc {
     time timestamp -> now
     vector3d position -> (733, 3047, 966)
     quaternion orientation -> .378, (0, -.925, 0)
     vector3f velocity -> (0, 0, 0)
     vector3f rotational_axis -> (0, 1, 0)
      float angular_speed -> 0
  }
}

Discovery of other object position updates

In the current revision the only way to discover objects have updated is to poll them for changes

MESSAGE "Camera Object" to 3277816b-a486-b9f2-4e07-62f39fb93413

message Header {
  optional uint32 source_port <- 0
  optional uuid destination_object -> 3277816b-a486-b9f2-4e07-62f39fb93413;
  optional uint32 destination_port -> 0
  
  optional uint64 id -> 1
}
message MessageBody {
  string message_names[0] -> "LocRequest"
  bytes message_body[0] -> message LocRequest {
 }
}

This is clearly not a sustainable solution, and will be replaced by an optional subscription service from either the Loc or Prox services Similar subscriptions can be setup for other persistence properties directly to an object---but these codes have not been fully integrated yet

Future Changes

• The initial proximity requests may return an initial ObjLoc information packet so that the object may be drawn sooner
• The initial proximity request might return a cached Mesh URI/persistence packet for faster appearance in the world
• LocRequests will be directed at the space rather than at an object
• Object updates may be subscribed to through a SubscriptionService class that uses out of band communication to get the messages through.