Add support for external event driven API

[njh]

• http://ruby-doc.org/stdlib-2.3.1/libdoc/observer/rdoc/Observable.html
• https://github.com/briandamaged/unobservable
• https://github.com/mikbe/eventable
• https://github.com/celluloid/celluloid
• https://github.com/shokai/event_emitter
• https://briandamaged.org/blog/ruby-event-handlers/

[skandragon]

+1

In my app, I need to do the following:

1. Periodically probe physical device endpoints for manual changes, and send MQTT messages.
2. Periodically scan for new devices, and send MQTT messages about them, as well as begin tracking them.
3. Process incoming MQTT state change requests and send them to physical devices

From what I can tell, I cannot do this using this mqtt library at this time. If I can, are there async examples?

[njh]

When I was writing the MQTT ruby gem I was unsure about the best approach to callbacks when messages were received; particularly when interacting with other code and libraries. So I opted to keep it synchronous/blocking to keep things simple.

At the time the premier async library for Ruby was EventMachine, so I wrote a gem for that: https://github.com/njh/ruby-em-mqtt

It still uses message parsing/generation from this gem. But I have found EventMachine somewhat frustrating to work with - particularly when there are errors or things go wrong.

If there are better/different approaches in Ruby now, please let me know...

[skandragon]

My current project just uses threads, which since the C ruby version has a global lock, is more of a way to allow cleaner code. If I put this under JRuby, I'd have to implement some simple locking where the two meet: my data structures.

I wasn't sure how safe it was to use a single shared MQTT client, so I went with two, one for the thing that monitors my light bulb's status (TP-Link LED dimmable bulbs, some RGB ones, and some AC switches) and another which watches for MQTT updates and sends commands to the bulbs.

"it works" so far :)

[andreimaxim]

Might I suggest the Observable module from the concurrent-ruby gem?

The concurrent-ruby gem is mature enough to be a requirement for Rails 5.0 (especially for thread safety features as thread_safe gem was moved into this one) and the Observable module is thread safe, as opposed to the standard library version. It could be interesting to do even more, like use actors (I'm currently working on a fork of this gem that does that and also automatically handles any connection issues but it's pre-pre-pre-alpha quality).

[njh]

Good suggestion. I have not looked at concurrent-ruby. Thanks @andreimaxim.

[skandragon]

I use RxScala at work -- Observables are handy.

[njh]

Or could go the libuv route:

• https://github.com/cotag/libuv

Which provides asynchronous callbacks, promises and all kinds of other fancy stuff, similar to node.js.

[jsaak]

Since in ruby 3.0 we have non-blocking fibers with a scheduler interface, it will be relatively easy to use that in this gem. I will do the changes for myself, to see how it works. Are you interested to merge it when i am done, or shall i fork?

[njh]

@jsaak yes, I would be very interested. Do you think it can be done without changing the API? It may make sense to fork, it it requires having code for both pre-ruby 3.0 and post-ruby 3.0.

I really need to get some PRs merged and a release made. After that I think the priority is the separate out the packet parsing/generating code, so that core can be used by other gems.

[jsaak]

Well, i decided to rewrite it from scratch, it is rewritten in different style which suits me better. Still maybe we can work something out. Have a look: https://github.com/jsaak/ruby-mqtt3 (it is a work in progress: QoS 1 and reconnect is implemented, today i plan to do QoS2)

[jsaak]

I have copy-pasted some functions from you, I hope you do not mind.

EDIT: i figured it out, so you can pass a string in any encoding, and it converts it to utf-8

there is one thing I do not understand: in the function encode_string you do:

def encode_string(str)
  str = str.to_s.encode('UTF-8')

  # Force to binary, when assembling the packet
  str.force_encoding('ASCII-8BIT')
  encode_short(str.bytesize) + str
end

https://github.com/njh/ruby-mqtt/blob/master/lib/mqtt/packet.rb#L248

What is the reason for this? i think this would be enough:

encode_short(str.bytesize) + str

or maybe

encode_short(str.bytesize) + str.force_encoding('ASCII-8BIT')