Protocol specification.

[kafkiansky]

Is there an existing proposal for this?

• [x] I have searched the existing proposals

Is your feature request related to a problem?

I want to implement a client to your broker.

Describe the solution you'd like

Will there be a protocol specification added?

Alternatives you considered

No response

[willhoy]

+1 Until there's something better you'd have to derive it from the bmqp module dox and the control message schema: https://github.com/bloomberg/blazingmq/blob/93e6426d474a4293d38244a9ada6dc8cbafe5711/src/groups/bmq/bmqp/bmqp_ctrlmsg.xsd

https://github.com/bloomberg/blazingmq/blob/93e6426d474a4293d38244a9ada6dc8cbafe5711/src/groups/bmq/bmqp/bmqp_protocol.h

These give you a good sense of the structure of the messages, but less so whats a valid response to a particular communication to the broker.

[quarter-note]

Hi @kafkiansky, thanks for your interest in the product! What @willhoy has pointed out are basically the two files which capture the wire protocol containing low level structs and control messages (encoded requests/responses).

However, these files don't explain the order of exchange of these structs/messages/requests/responses, as well as any contract between client and broker around that exchange.

We have not documented this yet. However, we have a sample producer and consumer implementation which does not use client library. Instead, it simply uses socket APIs to connect/read/write to the broker and sends/receives messages to/from the broker.
For example, one can see that after connecting to the broker, client library sends NegotiationMessage as the first message (see comment Step 3. Send the negotiation message to the Broker in the producer file), etc.

While this is not a formal spec, it should be enough to get you started to have a rudimentary working prototype.

I would also like to add that in order to provide a good experience, the C++ and Java client libraries are quite stateful. Asychronous nature of the APIs also adds complexity in the implementation. Generally speaking, implementing a batteries-included library is a non-trivial effort. However, we are happy to help you out if you decide to implement one.

Out of curiosity, what language do you have in mind for the client library?

[kafkiansky]

Hi, @quarter-note. Thanks for the explanation. I would like to implement the libraries in rust, go and php. I will try to use the information from those files you gave, or will watch tcpdump as a last resort.

[quarter-note]

Ok great, I will try to write a high level overview of the message flow and some other relevant details.

[kafkiansky]

Thank you so much for your help.

[quarter-note]

Hi @kafkiansky Just wanted to mention that we have started working on documenting the protocol. It's WIP but it can be found here. Hoping to wrap it up in the coming weeks.

[kafkiansky]

Hi, @quarter-note. Great, thank you. I've already started reverse-engineering your protocol using a Java library, but it would be much easier with a formal specification.

[quarter-note]

Hi @kafkiansky I have added some more details in the document. It's still WIP, but you may find newly added info useful.

[kafkiansky]

Hi @quarter-note. Yes, I am following the changes and going to continue development this weekend. Thank you for your work!

[kafkiansky]

@quarter-note, I didn't see any information about what order you write the bytes in: little endian or big endian?

[willhoy]

I would be very surprised if the bytes were not written in 'Network byte order' using the equivalent of hton*/ntoh* style functions at the low level.

[kafkiansky]

So it's BigEndian. Thanks for the hint, @willhoy.

[quarter-note]

Yes, it's big endian (or network byte order).

Some additional notes -- in Java, we have two wrapper classes ByteBufferInputStream and ByteBufferOutputStream which under the hood use java.nio.ByteBuffer, which take care of flipping the bytes when reading/writing integers, doubles, etc.

In C++, we use a BigEndian type from one of our helper libraries, which takes care of swapping the bytes seamlessly. As an example, here, we use BigEndianUint32 type to represent an unsigned int32 to capture the fragment and length fields of an EventHeader. Under the hood, various BigEndian classes use hton*/ntoh* routines as @willhoy mentioned.

[quarter-note]

Added some info in the doc.

[kafkiansky]

Hi, @quarter-note. While developing an SDK, I'm trying to run an OpenQueue request, but I'm getting an error {\"rId\":1,\"status\":{\"category\":\"E_UNKNOWN\",\"code\":-1,\"message\":-1,\"message\":\"Domain file '{\/etc\/local\/b/bmq\\\\/domains\\\\/myapp.json' doesn't exist\"}}}. What is that file? I couldn't find anything about such files in the documentation. The queue URI is bmq://myapp/queue.

Perhaps the bmqbrkr logs will be useful:

bmqbrkr_1  | 19AUG2023_18:08:49.300 (139933851313728) INFO mqba_domainresolver.cpp:190 Error reading the domain config file [domain: 'myapp', rc: -1, output:
bmqbrkr_1  | Domain file '/etc/local/bmq/domains/myapp.json' doesn't exist
bmqbrkr_1  | 19AUG2023_18:08:49.300 (139933851313728) WARN mqbblp_queuesessionmanager.cpp:114 #CLIENT_OPENQUEUE_FAILURE local:[email protected]:55432: Error while qualifying domain: [ category = E_UNKNOWN code = -1 message = "Domain file '/etc/local/bmq/domains/myapp.json' doesn't exist" ], request: [ rId = 1 choice = [ openQueue = [ handleParameters = [ uri = "bmq://myapp/queue" qId = 0 subIdInfo = NULL flags = 12 readCount = 0 writeCount = 1 adminCount = 0 ] ] ] ]
bmqbrkr_1  | 19AUG2023_18:08:49.300 (139933851313728) INFO mqba_clientsession.cpp:349 local:[email protected]:55432: Sending openQueue failure response: [ rId = 1 choice = [ status = [ category = E_UNKNOWN code = -1 message = "Domain file '/etc/local/bmq/domains/myapp.json' doesn't exist" ] ] ]
bmqbrkr_1  | 19AUG2023_18:08:49.303 (139933074130496) INFO mqbnet_tcpsessionfactory.cpp:734 TCPSessionFactory 'TCPInterface': OnClose channel [session: 'local:[email protected]:55432', channel: '172.19.0.1:55432#0x7f44c001c768', 0 active channels, status: [ Category = SUCCESS ]]

[678098]

Hi @kafkiansky

You try to use domain myapp for your queue. We don't have this domain in our sample domain configuration. We store domain configuration for each domain separately in its own json-file in the specified domains folder.

From the path /etc/local/bmq/domains/myapp.json probably you are launching bmqbrkr in a docker setup. In this setup we read domain configurations from this folder (for single node): https://github.com/bloomberg/blazingmq/tree/main/docker/single-node/config/domains

This folder is connected to the container as a volume here: https://github.com/bloomberg/blazingmq/blob/f4a4f55ae74caf24995ca540a0fbf70a33fbd9c3/docker/single-node/docker-compose.yaml#L8

So to fix this problem, you can either:

1. Use existing domain name, for priority mode for example, bmq.test.mem.priority instead of myapp (check the file bmq.test.mem.priority.json in the folder)
2. Or make your own domain configuration, by placing myapp.json to the domains folder. To start, you can simply copy and rename any existing domain configuration, this for example: https://github.com/bloomberg/blazingmq/blob/main/docker/single-node/config/domains/bmq.test.mem.priority.json

[678098]

Also some info about domains: https://bloomberg.github.io/blazingmq/docs/introduction/concepts/#domain

[kafkiansky]

@678098, thank you for the explanation.

[quarter-note]

@kafkiansky I have raised PR https://github.com/bloomberg/blazingmq/pull/95. Feel free to take a look and comment.

[kafkiansky]

Hello, @quarter-note. In the process of developing sdk on go, I am focusing on java sdk. There I noticed different packing of properties depending on the flag https://github.com/bloomberg/blazingmq-sdk-java/blob/main/bmq-sdk/src/main/java/com/bloomberg/bmq/impl/infr/proto/MessagePropertiesImpl.java#L416. What does this mean? Does this need to be supported or can only "newStyleProperties" be used?

[kafkiansky]

Hello, @678098. Maybe you know the answer to the question above?

[678098]

Hello, @678098. Maybe you know the answer to the question above?

Hi @kafkiansky! Only the new style message properties are needed. They are called "message properties V2" or "extended message properties". Some details about them are in this comment: https://github.com/bloomberg/blazingmq/issues/73#issuecomment-1668697386

[kafkiansky]

Hello, @678098. Maybe you know the answer to the question above?

Hi @kafkiansky! Only the new style message properties are needed. They are called "message properties V2" or "extended message properties". Some details about them are in this comment: #73 (comment)

Thank you, @678098.