General

Introducing

The design of any network application should start by describing messages sent between the consumer (client) and the producer (server). As better messages would be described, then more predictable would be an application in general.

CLIBRI allows describing messages in a strongly typed way to prevent any possible error related to interpretation and describe logic of communication between consumer and producer.

Using CLIBRI you will get your client-server (consumer-producer) solution just in 4 steps:

CLIBRI allows you to build stable network solutions really quickly and extend/scale them without huge additional efforts.

Besides, a protocol description and a workflow description together make your solution self-explainable. It would be enough to take a look into the workflow file (as usual a couple of hundreds of lines) to completely understand, what your application does and how it does.

You can combine platform and realizations: server (producer) on rust and client (consumer) on typescript. Or both on rust; or both on typescript. Considering CLIBRI generates a very similar API for all realizations it's always easy to work with any realization.

Protocol

To build a protocol implementation CLIBRI requires a protocol-description file. In general, it's just a text file, but we would recommend using extension *.prot for such kinds of files.

Below, you can see an example of a message.

struct Message {
    str field_str;
    u8 field_u8;
    u16 field_u16;
    u32 field_u32;
    u64 field_u64;
    i8 field_i8;
    i16 field_i16;
    i32 field_i32;
    i64 field_i64;
    f32 field_f32;
    f64 field_f64;
    bool field_bool;
}

The syntax is very simple and includes just a couple of possible entities.

struct Name_of_structure {
    field_type field_name;
    ...
    field_type field_name;
}

enum Name_of_enum {
    option_type option_name;
    ...
    option_type option_name;
}

group Name_of_group {
    struct Nested_structure {
        field_type field_name;
        ...
        field_type field_name;
    }
    enum Nested_enum {
        option_type option_name;
        ...
        option_type option_name;
    }
    group Nested_group {
        ...
    }
}

General entities

The whole protocol could be described using just three entities; which makes protocol transparent and understandable.

Primitive types

In the scope of structure and enum, you can define fields/options within the next primitive types.

Type Array definition Length Description
i8 i8[] 8-bit Signed integer
i16 i16[] 16-bit Signed integer
i32 i32[] 32-bit Signed integer
i64 i64[] 64-bit Signed integer
u8 u8[] 8-bit Unsigned integer
u16 u16[] 16-bit Unsigned integer
u32 u32[] 32-bit Unsigned integer
u64 u64[] 64-bit Unsigned integer
f32 f32[] 32-bit Floating-point number
f64 f64[] 64-bit Floating-point number
str str[] unlimited String value
bool bool[] 8-bit Boolean value

Structures and enums

The structure could have next kinds of fields:

To define a field as an optional symbol ? should be used. To define a field as an array, use brackets []. Definition of every single field should end with ;.

enum UserRole {
    Admin;
    User;
}

struct User {
    str nickname;
    UserRole role;          # Instance of enum
}

struct Message {
    User author;            # Instance of other structure
    str field_str?;         # Optional field
    u8 field_u8;            # Required field 
    u16[] field_array_u16;  # Array of values
}

The idea of enum - to have just one value from a list of possible values. In the previous example, you can see - structure User has a field role. role can be just one of two: Admin or User, but not together.

At the same time, CLIBRI protocol allows having values behind each option of enum (like in rust).

# Enum as a list of options
enum UserRole {
    Admin;
    User;
}

# Enum as a list of options with values
enum IncomeMessage {
    str text;
    u8[] bytes;
}

Groups

Group can be used to order and structure your protocol. Group cannot have any fields but can include definitions of structures and enums.

As soon as some group is created, you can refer to structures and enums in it.

group Messages {

    enum Content {
        str text;
        u8[] bytes;
    }

    struct Anonymous {
        Content message;
    }

    struct Authorized {
        str uuid;
        Content message;
    }
}

struct AllMessages {
    Messages.Anonymous[] anonymous;    # "." is used to define a path inside groups
    Messages.Authorized[] authorized;
}

More examples you can find on github

Protocol implementation

As soon as a scheme of the protocol is done you can implement it to get the necessary code-base. For a moment CLIBRI supports full implementation for rust and typescript (nodejs and browser).

To generate a code you need CLIBRI CLI tool. You can download it for multiple platforms.

clibri --src ./protocol.prot -rs ./rust/src/protocol.rs -ts ./typescript/src/protocol.ts -o --em
# --src path_to_protocol_scheme
# -rs path_to_rust_implementation_file
# -ts path_to_typescript_implementation_file

Here you can see a list all available keys of CLI.

More details about protocol implementation you would find for typescript and rust.

Workflow

As soon as we have a protocol for some project, it's already good because we can see: which kind of messages probably would be used for communication between parts of the solution. But does it give us an understanding of "how?" communication happens; what kind of logic is behind each message, each pair request - response? Unfortunately - no.

Let's see with an example. We would like to have a simple chat application. And for it, we would have the next protocol scheme.

enum UserRole { Admin; User; Manager; } group Events { struct UserConnected { str username; str uuid; } struct UserDisconnected { str username; str uuid; } struct Message { u64 timestamp; str user; str message; str uuid; } } group Beacons { struct LikeUser { str uuid; } struct LikeMessage { str uuid; } } group ServerEvents { struct UserKickOff { str reason?; str uuid; } struct UserAlert { str reason?; str uuid; } } group Message { struct Request { str user; str message; } struct Accepted { str uuid; } struct Denied { str reason; } struct Err { str error; } } group Messages { struct Message { u64 timestamp; str user; str uuid; str message; } struct Request { } struct Response { Message[] messages; } struct Err { str error; } } group UserLogin { struct Request { str username; } struct Accepted { str uuid; } struct Denied { str reason; } struct Err { str error; } } group UserInfo { struct Request { } struct Accepted { str browser; } struct Denied { str reason; } struct Err { str error; } } group Users { struct User { str name; str uuid; } struct Request { } struct Response { User[] users; } struct Err { str error; } }

The idea is simple, we grouped pairs "request-response" into groups. For example UserLogin.Request would expect a response with UserLogin.Accepted or UserLogin.Denied; or in case of some error UserLogin.Err.

Same logic we apply for other kinds of requests and responses. And it looks quite clean but still requires some clarifications. If we will give this protocol to some developer, we might hope he we will catch our idea with grouping, but better to give him some clarifications.

So how we can make our protocol self-explainable as much as possible?

CLIBRI has a solution for it - workflow. Workflow - is a scheme of communication between consumer/producer. Let see, how workflow could look like for our chat.

UserLogin.Request !UserLogin.Err { (Accept > UserLogin.Accepted) > Events.UserConnected; > Events.Message; (Deny > UserLogin.Denied); } Users.Request !Users.Err { (Users.Response); } Message.Request !Message.Err { (Accept > Message.Accepted) > Events.Message; (Deny > Message.Denied); } Messages.Request !Messages.Err { (Messages.Response); } # Broadcast for default event @disconnected { > Events.Message?; > Events.UserDisconnected; } # Broadcast for custom event @ServerEvents.UserKickOff { > Events.Message; > Events.UserDisconnected; } @ServerEvents.UserAlert { > Events.Message; > Events.UserConnected?; } # No response required messages from client. It's just events on producer side @beacons { > Beacons.LikeUser; > Beacons.LikeMessage; }

Workflow scheme could include next entities:

Requests / Responses

Any request should end with some response. Which kind of response, depends on the logic of our application. But obviously one of the possible responses - error. In terms of CLIBRI error means - no way to process a request. For example, if we are talking about user login procedures. An error could be - incorrect user's data, but a refusal of access - isn't an error, but one of the possible responses.

Well, logic of user login could be next:

Such kinds of possible responses are named in terms of CLIBRI - conclusions. Let's see to syntax.

RequestStructure !ErrorStructure { (ConclusionNameA > ResponseStructureA); ... (ConclusionNameB > ResponseStructureB); } # For example for login request: # 1. Accept: if user can be login - send UserLogin.Accepted back; # 2. Deny: if user can not be login - send UserLogin.Denied back; # 3. In case of error - send UserLogin.Err; UserLogin.Request !UserLogin.Err { (Accept > UserLogin.Accepted); (Deny > UserLogin.Denied); }

But sometimes with some incoming requests, we should not only respond to the sender but also notify some others. It's named - broadcasting. And we also can cover this logic.

RequestStructure !ErrorStructure { (ConclusionNameA > ResponseStructureA) > BroadcastStructureA; > BroadcastStructureB; ... (ConclusionNameB > ResponseStructureB) > BroadcastStructureC; > BroadcastStructureD; } # For example for login request: # 1. Accept: if user can be login - send UserLogin.Accepted back; # Also send to other users Events.UserConnected and Events.Message; # Events.UserConnected - notification about new user to update user's list # Events.Message - message into chat like "Noname join to chat" # 2. Deny: if user can not be login - send UserLogin.Denied back; # 3. In case of error - send UserLogin.Err; UserLogin.Request !UserLogin.Err { (Accept > UserLogin.Accepted) > Events.UserConnected; > Events.Message; (Deny > UserLogin.Denied); }

With such simple syntax, we've described the whole logic of the application.

In case if have just one possible response, we can use the short form:

RequestStructure !ErrorStructure { (ResponseStructure); } # Or with broadcasting RequestStructure !ErrorStructure { (ResponseStructure) > BroadcastStructureA; > BroadcastStructureB; }

If now you compare protocol definition and workflow description, you can see:

Well, workflow description gives an understanding of the application's logic.

Events

The trigger to send some messages to consumers in most cases would be some incoming request. But in some cases, it could be an event on the producer (server) side. To define such logic we can use events. Definition of event starts from symbol @.

# By some reason producer wants to disconnect (kickoff) some user # In addition server wants to notify other users: # - send notification to let users update users list # - send message into chat like "NoName user was ban" @ServerEvents.UserKickOff { > Events.Message; > Events.UserDisconnected; }

Syntax of events pretty simple:

@EventStructure { > BroadcastStructureA; ... > BroadcastStructureB; }

Take into account, as we are talking about consumer/producer model we always will have a couple of default events.

# Triggers as soon as new consumer connected @connected { > BroadcastStructureA; ... > BroadcastStructureB; } # Triggers as soon as some consumer disconnected @disconnected { > BroadcastStructureA; ... > BroadcastStructureB; }

These events are default and it means these events aren't bound with any structure from protocol.

Beacons

Beacon - is a one-way message from consumer to producer. It's useful in cases when the consumer isn't interested in any response but wants just to notify the producer about something.

Implementation of workflow considers a fact all beacons should be confirmed. It means, that a consumer will get not a response, but confirmation - beacon was received by a producer.

# To define list of possible beacons, keywork "@beacons" should be used @beacons { > BeaconStructureA; ... > BeaconStructureB; } # For example @beacons { > Beacons.LikeUser; > Beacons.LikeMessage; }

Broadcasting

In terms of CLIBRI, broadcasting means: sending a custom message to a not strictly defined list of consumers (event just to one consumer). For example, when we are talking about a response, a response always should be sent only to the sender of the request; but broadcast could be sent to others also.

Broadcasts could be triggered by incoming requests or by events on the producer side. You already have seen it before, but let's summary here all cases of usage.

# Broadcasting on incoming request # As soon request RequestStructure has been gotten producer will: # case "ConclusionNameA" # - response to sender with ResponseStructureA # - send message BroadcastStructureA to others consumers # - send message BroadcastStructureB to others consumers # case "ConclusionNameB" # - response to sender with ResponseStructureB # - send message BroadcastStructureC to others consumers # - send message BroadcastStructureD to others consumers RequestStructure !ErrorStructure { (ConclusionNameA > ResponseStructureA) > BroadcastStructureA; > BroadcastStructureB; ... (ConclusionNameB > ResponseStructureB) > BroadcastStructureC; > BroadcastStructureD; } # Broadcasting on events # As soon as event ProducerEventStructure is triggered, the producer # - send message BroadcastStructureA to consumers # - send message BroadcastStructureB to consumers @ProducerEventStructure { > BroadcastStructureA; > BroadcastStructureB; }

PlantUML diagrams

As soon as we've described the complete logic of communication between a consumer and a producer, we can easily build a diagram to render relations.

CLIBRI CLI has the necessary option to do it:

clibri --src ./protocol.prot --workflow ./protocol.workflow --puml ./plantuml.puml
# --src path_to_protocol_scheme
# --workflow path_to_workflow_scheme
# --puml path_to_puml_file to write diagram

Using our chat-example workflow scheme we will get next diagram

Let's take a look closer at the login request.

From the diagram we can get a full understanding of communication logic; we see:

Workflow implementation

Configuration

As soon as we are talking about a communication logic (but not just content - protocol) we need to give a little more data about workflow. First of all, it's an authorization. Each consumer should introduce itself somehow; at the same time, a producer should register consumer with might be some own representation. In terms of CLIBRI its:

If we back to our chat-example, it would be:

# Identification for chat protocol # Identification group in protocol description (not workflow description(!)) group Identification { # Next struture is used by consumer to introduce itself to producer struct SelfKey { str uuid?; u64 id?; str location?; } # Next strutucre is used by producer to register a consumer struct AssignedKey { str uuid?; bool auth?; } }

Take into account. Field uuid is required, and even you will not define it, CLIBRI will add it in any case.

As soon as we add necessary structures into a protocol, we can add a configuration section into the workflow description.

# Configuration section in workflow description &config { # Reference to self-key SelfKey: Identification.SelfKey; # Reference to assigned-key AssignedKey: Identification.AssignedKey; # Target language for producer code-base Producer: rust; # Target language for consumer code-base Consumer: rust; }

For CLIBRI doesn't matter where you will put the configuration section in the workflow file, but obviously, the beginning of the file looks better.

Configuration section starts with keyword &config.

Implementation

As soon as a scheme of the protocol and workflow are done you can implement it to get the necessary code-base. For a moment CLIBRI supports full implementation for rust and typescript (nodejs and browser).

To generate a code you need CLIBRI CLI tool. You can download it for multiple platforms.

clibri --src ./protocol.prot -wf ./protocol.workflow -cd ./consumer/src/consumer/ -pd ./producer/src/producer/
# --src path_to_protocol_scheme
# -wf path_to_workflow_file
# -cd path_to_consumer_codebase_folder
# -pd path_to_producer_codebase_folder