Typescript protocol implementation

General

Create simple empty npm project (based on typescript)

.
├── package.json
├── src
│   └── index.ts
├── prot
│   └── protocol.prot      <== protocol scheme, see below
├── tsconfig.json
└── tslint.json

Let's use for a demonstration next simple protocol scheme. Here is more about protocol syntax.

struct StructExampleA {
    str field_str;
    u32 field_u32;
    u64 field_u64;
    i8 field_i8;
    i16 field_i16;
    f64 field_f64;
    bool field_bool;
}

struct StructExampleB {
    str[] field_str;
    u8[] field_u8;
    i8[] field_i8;
    bool[] field_bool;
    StructExampleA field_struct;
    StructExampleA[] field_structs;
}

struct StructExampleC {
    str field_str?;
    u8 field_u8?;
    i16 field_i16?;
    i32 field_i32?;
    i64 field_i64?;
    bool field_bool?;
}

As soon as we have protocol scheme, let's generate code-base.

clibri --src ./prot/protocol.prot -ts ./src/protocol.ts -o --em

# --src path_to_protocol_scheme
# -o overwrite protocol.ts file if it already exists
# --em include all code to avoid using CLIBRI library
# -ts path_to_typescript_implementation_file

Now our solution looks like it:

.
├── package.json
├── src
│   ├── protocol.ts        <== protocol implementation
│   └── index.ts
├── prot
│   └── protocol.prot      
├── tsconfig.json
└── tslint.json

All our protocol's messages now are available in protocol.ts module.

import * as Protocol from "./protocol";

const message = new Protocol.StructExampleA({
    field_str: 'some string',
    field_u32: 1,
    field_u64: 2,
    field_i8: -1,
    field_i16: -2,
    field_f64: 0.1,
    field_bool: true,
});

So, every structure or enum would be a class in typescript; group - namespace.

Take into account, sometimes we need to do tests with some kind of dummy data, but if we have complex structures it might be quite annoying. You always can use static method defaults to get an instance of your structure with dummy data.

import * as Protocol from "./protocol";

const message = Protocol.StructExampleA.defaults();

Each instance of our struct has a couple of important methods:

Signature	Description
encode(): ArrayBufferLike	Returns bytes representation of structure
decode(from: ArrayBufferLike): Self \| Error	Makes attempt to create an instance of structure (class) from the given buffer
pack(sequence: number, uuid?: string): ArrayBufferLike	Packs instance of structure into a package ready to send and decode on another side

As you can see methods encode and pack look pretty the same. Both return bytes. The important difference is: the method pack prepares structure to send and if you are going to use CLIBRI reader, you should use a method pack. Because this method adds a header to the structure to correctly decode it after.

Let's see how CLIBRI reader works

import * as Protocol from "./protocol";

// Fill our buffer with some dummy data
const buffer = Buffer.concat([
    new Uint8Array(Protocol.StructExampleA.defaults().pack(1)),
    new Uint8Array(Protocol.StructExampleB.defaults().pack(2)),
    new Uint8Array(Protocol.StructExampleC.defaults().pack(3)),
]);

// Create reader
const reader: Protocol.BufferReaderMessages =
    new Protocol.BufferReaderMessages();

// Put data into reader
reader.chunk(buffer);

do {
    // Read message until all message would be read
    const received:
        | Protocol.IAvailableMessage
        | undefined = reader.next();
    if (received === undefined) {
        // No more messages in buffer
        break;
    }
    if (received.msg.StructExampleA !== undefined) {
        console.log(`StructExampleA has been gotten`);
    } else if (received.msg.StructExampleB !== undefined) {
        console.log(`StructExampleB has been gotten`);
    } else if (received.msg.StructExampleC !== undefined) {
        console.log(`StructExampleC has been gotten`);
    }
} while (true);

BufferReaderMessages has two major methods

Signature	Description
chunk(buffer: ArrayBufferLike)	Add data into reader buffer
next(): Protocol.IAvailableMessage \| undefined	Returns decoded message or undefined if buffer doesn't have any messages

In some cases, it might be you want to do some additional actions on a buffer before it would be sent. For example, compress and decompress data. For such kinds of purposes, you can use class PackingMiddleware.

All you need: create own implementation and init it once.

import * as Protocol from "./protocol";

export class Middleware extends Protocol.PackingMiddleware {
    public decode(
        buffer: ArrayBufferLike,
        id: number,
        sequence: number,
        uuid?: string
    ): ArrayBufferLike | Error {
        // Do some manipulations with buffer. For example compress.
        return buffer;
    }

    public encode(
        buffer: ArrayBufferLike,
        id: number,
        sequence: number,
        uuid?: string
    ): ArrayBufferLike | Error {
        // Do some manipulations with buffer. For example decompress.
        return buffer;
    }
}
// Create instance to register middleware
(new Middleware())

You don't need to register middleware anywhere in additional, CLIBRI will use global namespace to find it and use

More examples you can find on github

Packing vs encoding

As you already noticed there is a difference between packing and encoding, even both give bytes.

Packing adds a header to each message, which includes

id unique id of the message
sequence numeric sequence to bind messages between consumer and producer. Most relevant example: request - response. Both should have the same sequence to be bound.
timestamp time of packing message

At the same time encoding just gives you a byte representation of your message without any data, which would need to decode the message.