General
Introducing
The design of any network application should start by describing messages sent between the consumer (client) and the producer (server). As better messages would be described, then more predictable would be an application in general.
CLIBRI allows describing messages in a strongly typed way to prevent any possible error related to interpretation and describe logic of communication between consumer and producer.
Using CLIBRI you will get your client-server (consumer-producer) solution just in 4 steps:
- Step 1. Describe protocol (messages, events, broadcasts)
- Step 2. Describe workflow (relations between messages, events, broadcasts)
- Step 3. Generate code-base with CLIBRI
- Step 4. Use an almost ready solution (just put your code into handlers and run it)
CLIBRI allows you to build stable network solutions really quickly and extend/scale them without huge additional efforts.
Besides, a protocol description and a workflow description together make your solution self-explainable. It would be enough to take a look into the workflow file (as usual a couple of hundreds of lines) to completely understand, what your application does and how it does.
You can combine platform and realizations: server (producer) on rust and client (consumer) on typescript. Or both on rust; or both on typescript. Considering CLIBRI generates a very similar API for all realizations it's always easy to work with any realization.
Protocol
To build a protocol implementation CLIBRI requires a protocol-description file. In general, it's just a text file, but we would recommend using extension *.prot for such kinds of files.
Below, you can see an example of a message.
struct Message { str field_str; u8 field_u8; u16 field_u16; u32 field_u32; u64 field_u64; i8 field_i8; i16 field_i16; i32 field_i32; i64 field_i64; f32 field_f32; f64 field_f64; bool field_bool; }
The syntax is very simple and includes just a couple of possible entities.
struct Name_of_structure { field_type field_name; ... field_type field_name; } enum Name_of_enum { option_type option_name; ... option_type option_name; } group Name_of_group { struct Nested_structure { field_type field_name; ... field_type field_name; } enum Nested_enum { option_type option_name; ... option_type option_name; } group Nested_group { ... } }
General entities
The whole protocol could be described using just three entities; which makes protocol transparent and understandable.
- Structure. struct name { ... }. Structure it's an object with some amount of fields. The structure could have primitive fields (like int, strings, etc.) and references to other structures and enums.
- Enum. enum name { ... }. Enum's a strict and limited list of possible values. Each possible enum value is named "option". An option can be a reference to a primitive type, to structure, or to another enum.
- Group. group name { ... }. The main idea of a group is to collect structures or/and enums into some groups to get a more ordered and structured protocols. A group could include definitions of structures, enums, and nested groups.
Primitive types
In the scope of structure and enum, you can define fields/options within the next primitive types.
Type | Array definition | Length | Description |
---|---|---|---|
i8 | i8[] | 8-bit | Signed integer |
i16 | i16[] | 16-bit | Signed integer |
i32 | i32[] | 32-bit | Signed integer |
i64 | i64[] | 64-bit | Signed integer |
u8 | u8[] | 8-bit | Unsigned integer |
u16 | u16[] | 16-bit | Unsigned integer |
u32 | u32[] | 32-bit | Unsigned integer |
u64 | u64[] | 64-bit | Unsigned integer |
f32 | f32[] | 32-bit | Floating-point number |
f64 | f64[] | 64-bit | Floating-point number |
str | str[] | unlimited | String value |
bool | bool[] | 8-bit | Boolean value |
Structures and enums
The structure could have next kinds of fields:
- primitive typed field
- an instance of another structure
- an instance of enum
- optional field
- an array of primitive typed fields or array of other structures/enums
To define a field as an optional symbol ? should be used. To define a field as an array, use brackets []. Definition of every single field should end with ;.
enum UserRole { Admin; User; } struct User { str nickname; UserRole role; # Instance of enum } struct Message { User author; # Instance of other structure str field_str?; # Optional field u8 field_u8; # Required field u16[] field_array_u16; # Array of values }
The idea of enum - to have just one value from a list of possible values. In the previous example, you can see - structure User has a field role. role can be just one of two: Admin or User, but not together.
At the same time, CLIBRI protocol allows having values behind each option of enum (like in rust).
# Enum as a list of options enum UserRole { Admin; User; } # Enum as a list of options with values enum IncomeMessage { str text; u8[] bytes; }
Groups
Group can be used to order and structure your protocol. Group cannot have any fields but can include definitions of structures and enums.
As soon as some group is created, you can refer to structures and enums in it.
group Messages { enum Content { str text; u8[] bytes; } struct Anonymous { Content message; } struct Authorized { str uuid; Content message; } } struct AllMessages { Messages.Anonymous[] anonymous; # "." is used to define a path inside groups Messages.Authorized[] authorized; }
More examples you can find on github
Protocol implementation
As soon as a scheme of the protocol is done you can implement it to get the necessary code-base. For a moment CLIBRI supports full implementation for rust and typescript (nodejs and browser).
To generate a code you need CLIBRI CLI tool. You can download it for multiple platforms.
clibri --src ./protocol.prot -rs ./rust/src/protocol.rs -ts ./typescript/src/protocol.ts -o --em # --src path_to_protocol_scheme # -rs path_to_rust_implementation_file # -ts path_to_typescript_implementation_file
Here you can see a list all available keys of CLI.
More details about protocol implementation you would find for typescript and rust.
Workflow
As soon as we have a protocol for some project, it's already good because we can see: which kind of messages probably would be used for communication between parts of the solution. But does it give us an understanding of "how?" communication happens; what kind of logic is behind each message, each pair request - response? Unfortunately - no.
Let's see with an example. We would like to have a simple chat application. And for it, we would have the next protocol scheme.
enum UserRole {
Admin;
User;
Manager;
}
group Events {
struct UserConnected {
str username;
str uuid;
}
struct UserDisconnected {
str username;
str uuid;
}
struct Message {
u64 timestamp;
str user;
str message;
str uuid;
}
}
group Beacons {
struct LikeUser {
str uuid;
}
struct LikeMessage {
str uuid;
}
}
group ServerEvents {
struct UserKickOff {
str reason?;
str uuid;
}
struct UserAlert {
str reason?;
str uuid;
}
}
group Message {
struct Request {
str user;
str message;
}
struct Accepted {
str uuid;
}
struct Denied {
str reason;
}
struct Err {
str error;
}
}
group Messages {
struct Message {
u64 timestamp;
str user;
str uuid;
str message;
}
struct Request {
}
struct Response {
Message[] messages;
}
struct Err {
str error;
}
}
group UserLogin {
struct Request {
str username;
}
struct Accepted {
str uuid;
}
struct Denied {
str reason;
}
struct Err {
str error;
}
}
group UserInfo {
struct Request {
}
struct Accepted {
str browser;
}
struct Denied {
str reason;
}
struct Err {
str error;
}
}
group Users {
struct User {
str name;
str uuid;
}
struct Request {
}
struct Response {
User[] users;
}
struct Err {
str error;
}
}
The idea is simple, we grouped pairs "request-response" into groups. For example UserLogin.Request would expect a response with UserLogin.Accepted or UserLogin.Denied; or in case of some error UserLogin.Err.
Same logic we apply for other kinds of requests and responses. And it looks quite clean but still requires some clarifications. If we will give this protocol to some developer, we might hope he we will catch our idea with grouping, but better to give him some clarifications.
So how we can make our protocol self-explainable as much as possible?
CLIBRI has a solution for it - workflow. Workflow - is a scheme of communication between consumer/producer. Let see, how workflow could look like for our chat.
UserLogin.Request !UserLogin.Err {
(Accept > UserLogin.Accepted) > Events.UserConnected;
> Events.Message;
(Deny > UserLogin.Denied);
}
Users.Request !Users.Err {
(Users.Response);
}
Message.Request !Message.Err {
(Accept > Message.Accepted) > Events.Message;
(Deny > Message.Denied);
}
Messages.Request !Messages.Err {
(Messages.Response);
}
# Broadcast for default event
@disconnected {
> Events.Message?;
> Events.UserDisconnected;
}
# Broadcast for custom event
@ServerEvents.UserKickOff {
> Events.Message;
> Events.UserDisconnected;
}
@ServerEvents.UserAlert {
> Events.Message;
> Events.UserConnected?;
}
# No response required messages from client. It's just events on producer side
@beacons {
> Beacons.LikeUser;
> Beacons.LikeMessage;
}
Workflow scheme could include next entities:
- request. Description of pair "request-response".
- event. Description of events. An event could be triggered only on the producer side. As soon as even triggered we might be want to send some message (broadcast) to consumers.
- beacons. List of possible beacons from the consumer side. In some cases, the consumer wants just to notify the producer about something. The consumer doesn't need any response, this is a one-way message.
Requests / Responses
Any request should end with some response. Which kind of response, depends on the logic of our application. But obviously one of the possible responses - error. In terms of CLIBRI error means - no way to process a request. For example, if we are talking about user login procedures. An error could be - incorrect user's data, but a refusal of access - isn't an error, but one of the possible responses.
Well, logic of user login could be next:
- Request to login
- Possible response: allow
- Possible response: deny
- Possible error: invalid request
Such kinds of possible responses are named in terms of CLIBRI - conclusions. Let's see to syntax.
RequestStructure !ErrorStructure {
(ConclusionNameA > ResponseStructureA);
...
(ConclusionNameB > ResponseStructureB);
}
# For example for login request:
# 1. Accept: if user can be login - send UserLogin.Accepted back;
# 2. Deny: if user can not be login - send UserLogin.Denied back;
# 3. In case of error - send UserLogin.Err;
UserLogin.Request !UserLogin.Err {
(Accept > UserLogin.Accepted);
(Deny > UserLogin.Denied);
}
But sometimes with some incoming requests, we should not only respond to the sender but also notify some others. It's named - broadcasting. And we also can cover this logic.
RequestStructure !ErrorStructure {
(ConclusionNameA > ResponseStructureA) > BroadcastStructureA;
> BroadcastStructureB;
...
(ConclusionNameB > ResponseStructureB) > BroadcastStructureC;
> BroadcastStructureD;
}
# For example for login request:
# 1. Accept: if user can be login - send UserLogin.Accepted back;
# Also send to other users Events.UserConnected and Events.Message;
# Events.UserConnected - notification about new user to update user's list
# Events.Message - message into chat like "Noname join to chat"
# 2. Deny: if user can not be login - send UserLogin.Denied back;
# 3. In case of error - send UserLogin.Err;
UserLogin.Request !UserLogin.Err {
(Accept > UserLogin.Accepted) > Events.UserConnected;
> Events.Message;
(Deny > UserLogin.Denied);
}
With such simple syntax, we've described the whole logic of the application.
In case if have just one possible response, we can use the short form:
RequestStructure !ErrorStructure {
(ResponseStructure);
}
# Or with broadcasting
RequestStructure !ErrorStructure {
(ResponseStructure) > BroadcastStructureA;
> BroadcastStructureB;
}
If now you compare protocol definition and workflow description, you can see:
- from protocol, you get an understanding of the message. You see group Message and Event.Message. And probably you can try to guess, when and how the message would be sent.
- but the workflow description gives you exact information about logic and you can see - the message would be sent also as soon as some new user comes.
Well, workflow description gives an understanding of the application's logic.
Events
The trigger to send some messages to consumers in most cases would be some incoming request. But in some cases, it could be an event on the producer (server) side. To define such logic we can use events. Definition of event starts from symbol @.
# By some reason producer wants to disconnect (kickoff) some user
# In addition server wants to notify other users:
# - send notification to let users update users list
# - send message into chat like "NoName user was ban"
@ServerEvents.UserKickOff {
> Events.Message;
> Events.UserDisconnected;
}
Syntax of events pretty simple:
@EventStructure {
> BroadcastStructureA;
...
> BroadcastStructureB;
}
Take into account, as we are talking about consumer/producer model we always will have a couple of default events.
# Triggers as soon as new consumer connected
@connected {
> BroadcastStructureA;
...
> BroadcastStructureB;
}
# Triggers as soon as some consumer disconnected
@disconnected {
> BroadcastStructureA;
...
> BroadcastStructureB;
}
These events are default and it means these events aren't bound with any structure from protocol.
Beacons
Beacon - is a one-way message from consumer to producer. It's useful in cases when the consumer isn't interested in any response but wants just to notify the producer about something.
Implementation of workflow considers a fact all beacons should be confirmed. It means, that a consumer will get not a response, but confirmation - beacon was received by a producer.
# To define list of possible beacons, keywork "@beacons" should be used
@beacons {
> BeaconStructureA;
...
> BeaconStructureB;
}
# For example
@beacons {
> Beacons.LikeUser;
> Beacons.LikeMessage;
}
Broadcasting
In terms of CLIBRI, broadcasting means: sending a custom message to a not strictly defined list of consumers (event just to one consumer). For example, when we are talking about a response, a response always should be sent only to the sender of the request; but broadcast could be sent to others also.
Broadcasts could be triggered by incoming requests or by events on the producer side. You already have seen it before, but let's summary here all cases of usage.
# Broadcasting on incoming request
# As soon request RequestStructure has been gotten producer will:
# case "ConclusionNameA"
# - response to sender with ResponseStructureA
# - send message BroadcastStructureA to others consumers
# - send message BroadcastStructureB to others consumers
# case "ConclusionNameB"
# - response to sender with ResponseStructureB
# - send message BroadcastStructureC to others consumers
# - send message BroadcastStructureD to others consumers
RequestStructure !ErrorStructure {
(ConclusionNameA > ResponseStructureA) > BroadcastStructureA;
> BroadcastStructureB;
...
(ConclusionNameB > ResponseStructureB) > BroadcastStructureC;
> BroadcastStructureD;
}
# Broadcasting on events
# As soon as event ProducerEventStructure is triggered, the producer
# - send message BroadcastStructureA to consumers
# - send message BroadcastStructureB to consumers
@ProducerEventStructure {
> BroadcastStructureA;
> BroadcastStructureB;
}
PlantUML diagrams
As soon as we've described the complete logic of communication between a consumer and a producer, we can easily build a diagram to render relations.
CLIBRI CLI has the necessary option to do it:
clibri --src ./protocol.prot --workflow ./protocol.workflow --puml ./plantuml.puml # --src path_to_protocol_scheme # --workflow path_to_workflow_scheme # --puml path_to_puml_file to write diagram
Using our chat-example workflow scheme we will get next diagram
Let's take a look closer at the login request.
From the diagram we can get a full understanding of communication logic; we see:
- producer could respond with one of 2 possible responses
- also producer could respond with an error
- and at last point, the producer will send some broadcast messages only in case of response Accept
Workflow implementation
Configuration
As soon as we are talking about a communication logic (but not just content - protocol) we need to give a little more data about workflow. First of all, it's an authorization. Each consumer should introduce itself somehow; at the same time, a producer should register consumer with might be some own representation. In terms of CLIBRI its:
- self-key structure - structure, which is used by a consumer to introduce itself to producer; self-key can be changed only by request of a consumer.
- assigned-key structure - structure, which is used by a producer to register each consumer; a producer cannot change self-key of a consumer but can change assigned-key.
If we back to our chat-example, it would be:
# Identification for chat protocol
# Identification group in protocol description (not workflow description(!))
group Identification {
# Next struture is used by consumer to introduce itself to producer
struct SelfKey {
str uuid?;
u64 id?;
str location?;
}
# Next strutucre is used by producer to register a consumer
struct AssignedKey {
str uuid?;
bool auth?;
}
}
Take into account. Field uuid is required, and even you will not define it, CLIBRI will add it in any case.
As soon as we add necessary structures into a protocol, we can add a configuration section into the workflow description.
# Configuration section in workflow description
&config {
# Reference to self-key
SelfKey: Identification.SelfKey;
# Reference to assigned-key
AssignedKey: Identification.AssignedKey;
# Target language for producer code-base
Producer: rust;
# Target language for consumer code-base
Consumer: rust;
}
For CLIBRI doesn't matter where you will put the configuration section in the workflow file, but obviously, the beginning of the file looks better.
Configuration section starts with keyword &config.
Implementation
As soon as a scheme of the protocol and workflow are done you can implement it to get the necessary code-base. For a moment CLIBRI supports full implementation for rust and typescript (nodejs and browser).
To generate a code you need CLIBRI CLI tool. You can download it for multiple platforms.
clibri --src ./protocol.prot -wf ./protocol.workflow -cd ./consumer/src/consumer/ -pd ./producer/src/producer/ # --src path_to_protocol_scheme # -wf path_to_workflow_file # -cd path_to_consumer_codebase_folder # -pd path_to_producer_codebase_folder