Add asyncio support

[GimmickNG]

With reference to my comment in #5, I've added asyncio support for TCP and limited support for Serial RTU. Right now, async serial support is TBD but I've left the class in just in case anyone else wants to try modifying it themselves. To avoid code duplication, I've based the async classes off their synchronous equivalents, and only modified parts when I couldn't cleanly link the two.

I haven't tested the results yet, but I expect it to work for the most part. So I guess until later, this is still very much as "use at your own risk" situation since it could be rather buggy now.

[brainelectronics]

Thank you @GimmickNG for your PR 🎊

I need a deeper review and testing the next days but the first impression is very good 👍🏻

• [ ] Check CI run results
• [ ] missing semver valid Changelog entry

[brainelectronics]

Please mention #5 and #11 in the Changelog entry

[GimmickNG]

Thanks for the review. I'll make the changes required (and I also spotted some errors when I tried using the library, whose fixes I'll add here) in a week or so, when I should have more time available.

[GimmickNG]

I'm not sure why Github doesn't recognize that the async folder was renamed to asynchronous in my latest revision. Oh well. Here are the changes to save time on reviewing:

• Fixed flake8 errors in both synchronous and asynchronous versions (line too long, incorrect indent level, etc.)
• Fixed async serial version to be more like async TCP. Requires testing as I do not have a board with UART
• Breaking change: Renamed Serial class to RTUServer to match TCPServer terminology (since functions are similar)
• Renamed async folder to asynchronous as importing the package resulted in a syntax error (due to it being a reserved name)
• Fixed errors with async TCP, encountered during testing on Pycopy/Micropython for Ubuntu. Should work for the most part.
• Added changelog entries (but since this was a WIP, only mentioned "async support" for all the above changes under the "Added" category)

[beyonlo]

Hello. I'm happy that we are closer to have the async feature ready on the ModBus lib :partying_face:

I would like to say that I will happy to help on the tests with the serial version (RTU) that require a board with UART. I have one board with RS485 already working! As soon this PR will be merged I can to help do the tests :smiley:

[brainelectronics]

@beyonlo could you test the first release candidate of this lib with Async support?

https://test.pypi.org/project/micropython-modbus/2.4.0rc39.dev56/

[beyonlo]

Hello @brainelectronics

@beyonlo could you test the first release candidate of this lib with Async support?

Yes, of course I can do that.

Are there some example to run the Slave TCP and Slave RTU using the uasyncio, or a simple doc how to use/start it?

https://test.pypi.org/project/micropython-modbus/2.4.0rc39.dev56/

I downloaded this version, but it seems not to have the uasyncio support.

pi@pi:~/Downloads/micropython-modbus-2.4.0rc39.dev56$ ls -la
total 32
drwxrwxr-x 4 pi pi  4096 mar  6 10:38 .
drwxr-xr-x 5 pi pi 20480 mar  6 10:38 ..
drwxrwxr-x 2 pi pi  4096 mar  6 10:38 micropython_modbus.egg-info
drwxrwxr-x 2 pi pi  4096 mar  6 10:46 umodbus
pi@pi:~/Downloads/micropython-modbus-2.4.0rc39.dev56$ cd umodbus/
pi@pi:~/Downloads/micropython-modbus-2.4.0rc39.dev56/umodbus$ ls -la
total 136
drwxrwxr-x 2 pi pi  4096 mar  6 10:46 .
drwxrwxr-x 4 pi pi  4096 mar  6 10:38 ..
-rw-r--r-- 1 pi pi 14741 mar  6 05:04 common.py
-rw-r--r-- 1 pi pi  6785 mar  6 05:04 const.py
-rw-r--r-- 1 pi pi 14778 mar  6 05:04 functions.py
-rw-r--r-- 1 pi pi   108 mar  6 05:04 __init__.py
-rw-r--r-- 1 pi pi 36148 mar  6 05:04 modbus.py
-rw-r--r-- 1 pi pi 17275 mar  6 05:04 serial.py
-rw-r--r-- 1 pi pi  1304 mar  6 05:04 sys_imports.py
-rw-r--r-- 1 pi pi 15704 mar  6 05:04 tcp.py
-rw-r--r-- 1 pi pi  2121 mar  6 05:04 typing.py
-rw-r--r-- 1 pi pi   140 mar  6 05:07 version.py
pi@pi:~/Downloads/micropython-modbus-2.4.0rc39.dev56/umodbus$ cat version.py 
#!/usr/bin/env python3
# -*- coding: UTF-8 -*-

__version_info__ = ("2", "4", "0")
__version__ = '.'.join(__version_info__) + '-rc39.dev56'
pi@pi:~/Downloads/micropython-modbus-2.4.0rc39.dev56/umodbus$ grep -i uasyncio *
pi@pi:~/Downloads/micropython-modbus-2.4.0rc39.dev56/umodbus$

Thank you!

[GimmickNG]

@beyonlo I am currently writing a few examples for the async clients and servers. However, I noticed that when writing the client and server examples for async RTU, some features in TCP were not present in RTU and had to be implemented manually (such as serve_forever()). Also, starting an async RTU client was made confusing since it required creating an AsyncRTUServer instance, which makes no sense to the end user.

@brainelectronics I am making changes to make the async version more consistent with the sync one for the RTU implementation - namely, changing the regular CommonRTUFunctions to RTUServer and using this to create an AsyncRTU (client) and a proper AsyncRTUServer class, which will be used by AsyncModbusRTU to start an RTU server. What are your thoughts on this, with respect to e.g. backwards compatibility?

[brainelectronics]

@beyonlo best finding so far 😂 I've fixed the packaging code, let's try again with 2.4.0rc40.dev56

@GimmickNG could you please update the setup.py as follows

    packages=['umodbus', 'umodbus/asynchronous'],

Regarding backwards compability: As long as the following imports and the top level function names do not change in the sync part of the lib, I'm open to many changes as long as they are related to this PR of course

from umodbus.serial import ModbusRTU
from umodbus.serial import Serial as ModbusRTUMaster
from umodbus.tcp import ModbusTCP
from umodbus.tcp import TCP as ModbusTCPMaster

[beyonlo]

@GimmickNG Thanks for your effort to support uasyncio on this lib. I will wait your changes on the uasyncio version, as well the examples for the uasyncio clients and servers to start the tests!

Will be all features of this lib available/works on the uasyncio version too, including the callbacks functions when Master set/get the address on the Slaves?

[GimmickNG]

@beyonlo Since the async version is based off/depends on the sync version, the two should be fairly similar (with the exception of asyncio.run() needed to start the asyncio loop), so yes the callbacks should be supported --- let me know if they aren't.

Also, check async_examples.py for the async TCP/RTU examples. I haven't verified these though so there may be some errors, but I don't expect any initially.

@brainelectronics I split off the server/slave functionality of Serial into a separate RTUServer class since it looked overloaded in comparison to TCP/TCPServer. I tried to keep the user-facing interface as close to the original as possible, so the ModbusRTU class remains the same --- it now uses RTUServer instead of Serial, and Serial is now an RTU client only.

[beyonlo]

Hello @GimmickNG

@beyonlo Since the async version is based off/depends on the sync version, the two should be fairly similar (with the exception of asyncio.run() needed to start the asyncio loop), so yes the callbacks should be supported --- let me know if they aren't.

Understood!

Also, check async_examples.py for the async TCP/RTU examples. I haven't verified these though so there may be some errors, but I don't expect any initially.

I followed your examples to do a simple test running just the Slave TCP using the 2.4.0rc40.dev56, but I have errors. I think that reason is because you did that changes and waiting for the @brainelectronics review and to create a new package.

start_tcp_server.py:

try:
    import uasyncio as asyncio
except ImportError:
    import asyncio

from umodbus.asynchronous.tcp import AsyncModbusTCP, AsyncTCP
from umodbus.asynchronous.serial import AsyncModbusRTU, AsyncSerial

async def start_tcp_server(host, port, backlog):
    server = AsyncModbusTCP()
    await server.bind(local_ip=host, local_port=port, max_connections=backlog)
    await server.serve_forever()

def run_tcp_test(host, port, backlog):
    asyncio.run(start_tcp_server(host, port, backlog))

run_tcp_test('192.168.43.35', 502, 2)

$ mpremote run start_tcp_server.py
Traceback (most recent call last):
  File "<stdin>", line 6, in <module>
  File "umodbus/asynchronous/tcp.py", line 21, in <module>
  File "umodbus/asynchronous/modbus.py", line 20, in <module>
  File "umodbus/modbus.py", line 27, in <module>
  File "umodbus/modbus.py", line 672, in Modbus
TypeError: function takes 0 positional arguments but 1 were given

Question: I need to run the Slave TCP and Slave RTU simultaneously in my ESP32-S3. So, how I do to start both (start_tcp_server and start_rtu_server) if just one await server.serve_forever() can to run at same time?

[GimmickNG]

@beyonlo

$ mpremote run start_tcp_server.py
Traceback (most recent call last):
  File "<stdin>", line 6, in <module>
  File "umodbus/asynchronous/tcp.py", line 21, in <module>
  File "umodbus/asynchronous/modbus.py", line 20, in <module>
  File "umodbus/modbus.py", line 27, in <module>
  File "umodbus/modbus.py", line 672, in Modbus
TypeError: function takes 0 positional arguments but 1 were given

Thanks for letting me know. I'll try with my local version and get back to you later if it's because of the old version or if it's an error in the new code as well.

Question: I need to run the Slave TCP and Slave RTU simultaneously in my ESP32-S3. So, how I do to start both (start_tcp_server and start_rtu_server) if just one await server.serve_forever() can to run at same time?

I'm not sure, but from what I've seen elsewhere, I don't think you need await server.serve_forever() if you are running multiple servers. The serve_forever() seems to be made to ensure that the server task is not cancelled by the program finishing everything else prematurely - so you could (in theory) just have bind(), and then process your other server functions. Something like this (untested):

try:
    import uasyncio as asyncio
except ImportError:
    import asyncio

from umodbus.asynchronous.tcp import AsyncModbusTCP, AsyncTCP
from umodbus.asynchronous.serial import AsyncModbusRTU, AsyncSerial

async def start_tcp_server(host, port, backlog):
    server = AsyncModbusTCP()
    await server.bind(local_ip=host, local_port=port, max_connections=backlog)
    return server

async def start_rtu_server(addr, **kwargs):
    server = AsyncModbusRTU(addr, **kwargs)
    return server

async def run_both():
    tcp_server = await start_tcp_server()
    rtu_server = await start_rtu_server()
    # either (if you are running 2 servers and also some extra tasks)
    while True:
        # other tasks here
        await rtu_server.process()
    # and then cleanup tcp server once application exits
    tcp_server.server_close()
    
    # or (if only the servers are running)
    # await asyncio.gather(tcp_server.serve_forever(), rtu_server.serve_forever())

asyncio.run(run_both())

[brainelectronics]

This is the next release candidate out for testing 😄 Thank you @beyonlo for testing and thank you @GimmickNG for providing

https://test.pypi.org/project/micropython-modbus/2.4.0rc42.dev56/

[beyonlo]

This is the next release candidate out for testing smile Thank you @beyonlo for testing and thank you @GimmickNG for providing

https://test.pypi.org/project/micropython-modbus/2.4.0rc42.dev56/

Hi all :)

Just to notice you, this version still have the same error:

start_tcp_server.py:

try:
    import uasyncio as asyncio
except ImportError:
    import asyncio

import os

print(f'MicroPython version: {os.uname()}')
from umodbus import version
print(f'Running ModBus version: {version.__version__}')

from umodbus.asynchronous.tcp import AsyncModbusTCP, AsyncTCP
from umodbus.asynchronous.serial import AsyncModbusRTU, AsyncSerial

async def start_tcp_server(host, port, backlog):
    server = AsyncModbusTCP()
    await server.bind(local_ip=host, local_port=port, max_connections=backlog)
    await server.serve_forever()

def run_tcp_test(host, port, backlog):
    asyncio.run(start_tcp_server(host, port, backlog))

run_tcp_test('192.168.43.143', 502, 2)

Output:

$ mpremote run start_tcp_server.py 
MicroPython version: (sysname='esp32', nodename='esp32', release='1.19.1', version='v1.19.1-966-g05bb26010 on 2023-03-13', machine='ESP32S3 module (spiram) with ESP32S3')
Running ModBus version: 2.4.0-rc42.dev56
Traceback (most recent call last):
  File "<stdin>", line 12, in <module>
  File "umodbus/asynchronous/tcp.py", line 21, in <module>
  File "umodbus/asynchronous/modbus.py", line 20, in <module>
  File "umodbus/modbus.py", line 27, in <module>
  File "umodbus/modbus.py", line 672, in Modbus
TypeError: function takes 0 positional arguments but 1 were given

[brainelectronics]

@beyonlo thanks for checking, I get the same result on my board locally.

I just merged #64 to run the Docker based tests also on external PRs, could you @GimmickNG update your branch to get this changes in and then have the tests running as well.

[GimmickNG]

@brainelectronics sure. I think the reason for the error (assuming line 672 of modbus.py is correct) is probably because of the overload decorator in typing.py. I didn't get that error during testing because I was using the typing library of pycopy, not the file. I'll change the imports to use the local typing.py import and test to see if I encounter that error.

[GimmickNG]

Looks like it was indeed because of the error with the @overload decorator in typing.py. The async_examples should work now - I managed to get the basic tcp client and server test running on my machine.

[beyonlo]

Hello @GimmickNG! As @brainelectronics do not created a new package with your modification, and I see that your changes is just the + def overload(fun): (Am I right?), so I changed that.

Well, after I changed that on the https://test.pypi.org/project/micropython-modbus/2.4.0rc42.dev56/, the example below run without error now :) Sorry for my ignorance, but I have no idea how to integrate the register definitions, callbacks, etc to run with the async version. With the sync version there is the client.setup_registers(registers=register_definitions) that call the register_definitions, but how that is called by async version? Bellow there is as well a very simple sync example of tcp_client_example.py (based from the tcp_client_example.py ) that I trying to port to use with async version. Could you please, help me how to port that or point me a example of async version using the register_definitions?

My intention is start testing the ModBus TCP Slave and after ModBus RTU Slave, and finally the Masters RTU and TCP. All in async version :)

Thank you so much!

start_tcp_server.py: (async ModBus TCP Slave)

try:
    import uasyncio as asyncio
except ImportError:
    import asyncio

import os

print(f'MicroPython version: {os.uname()}')
from umodbus import version
print(f'Running ModBus version: {version.__version__}')

from umodbus.asynchronous.tcp import AsyncModbusTCP, AsyncTCP
from umodbus.asynchronous.serial import AsyncModbusRTU, AsyncSerial

async def start_tcp_server(host, port, backlog):
    server = AsyncModbusTCP()
    await server.bind(local_ip=host, local_port=port, max_connections=backlog)
    await server.serve_forever()

def run_tcp_test(host, port, backlog):
    asyncio.run(start_tcp_server(host, port, backlog))

run_tcp_test('192.168.43.143', 502, 2)

Output:

$ mpremote run start_tcp_server.py 
MicroPython version: (sysname='esp32', nodename='esp32', release='1.19.1', version='v1.19.1-966-g05bb26010 on 2023-03-13', machine='ESP32S3 module (spiram) with ESP32S3')
Running ModBus version: 2.4.0-rc42.dev56

tcp_client_example.py: (sync ModBus TCP Slave)

#!/usr/bin/env python3
# -*- coding: UTF-8 -*-

"""
Main script

Do your stuff here, this file is similar to the loop() function on Arduino

Create a Modbus TCP client (slave) which can be requested for data or set with
specific values by a host device.

The TCP port and IP address can be choosen freely. The register definitions of
the client can be defined by the user.
"""

from umodbus import version
print('Running ModBus version: {}'.format(version.__version__))

# system packages
import time

# import modbus client classes
from umodbus.tcp import ModbusTCP

IS_DOCKER_MICROPYTHON = False
try:
    import network
except ImportError:
    IS_DOCKER_MICROPYTHON = True
    import json


# ===============================================
if IS_DOCKER_MICROPYTHON is False:
    # connect to a network
    station = network.WLAN(network.STA_IF)
    if station.active() and station.isconnected():
        station.disconnect()
        time.sleep(1)
    station.active(False)
    time.sleep(1)
    station.active(True)

    # station.connect('SSID', 'PASSWORD')
    station.connect('Aqui', 'jupi6124')
    time.sleep(1)

    while True:
        print('Waiting for WiFi connection...')
        if station.isconnected():
            print('Connected to WiFi.')
            print(station.ifconfig())
            break
        time.sleep(2)

# ===============================================
# TCP Slave setup
tcp_port = 502              # port to listen to

if IS_DOCKER_MICROPYTHON:
    local_ip = '172.24.0.2'     # static Docker IP address
else:
    # set IP address of the MicroPython device explicitly
    # local_ip = '192.168.4.1'    # IP address
    # or get it from the system after a connection to the network has been made
    local_ip = station.ifconfig()[0]

# ModbusTCP can get TCP requests from a host device to provide/set data
client = ModbusTCP()
is_bound = False

# check whether client has been bound to an IP and port
is_bound = client.get_bound_status()

if not is_bound:
    client.bind(local_ip=local_ip, local_port=tcp_port)


def my_coil_set_cb(reg_type, address, val):
    print('Custom callback, called on setting {} at {} to: {}'.
          format(reg_type, address, val))


def my_coil_get_cb(reg_type, address, val):
    print('Custom callback, called on getting {} at {}, currently: {}'.
          format(reg_type, address, val))


def my_holding_register_set_cb(reg_type, address, val):
    print('Custom callback, called on setting {} at {} to: {}'.
          format(reg_type, address, val))


def my_holding_register_get_cb(reg_type, address, val):
    print('Custom callback, called on getting {} at {}, currently: {}'.
          format(reg_type, address, val))


def my_discrete_inputs_register_get_cb(reg_type, address, val):
    print('Custom callback, called on getting {} at {}, currently: {}'.
          format(reg_type, address, val))


def my_inputs_register_get_cb(reg_type, address, val):
    # usage of global isn't great, but okay for an example
    global client

    print('Custom callback, called on getting {} at {}, currently: {}'.
          format(reg_type, address, val))

    # any operation should be as short as possible to avoid response timeouts
    new_val = val[0] + 1

    # It would be also possible to read the latest ADC value at this time
    # adc = machine.ADC(12)     # check MicroPython port specific syntax
    # new_val = adc.read()

    client.set_ireg(address=address, value=new_val)
    print('Incremented current value by +1 before sending response')


def reset_data_registers_cb(reg_type, address, val):
    # usage of global isn't great, but okay for an example
    global client
    global register_definitions

    print('Resetting register data to default values ...')
    client.setup_registers(registers=register_definitions)
    print('Default values restored')


# commond slave register setup, to be used with the Master example above
register_definitions = {
    "COILS": {
        "RESET_REGISTER_DATA_COIL": {
            "register": 42,
            "len": 1,
            "val": 0
        },
        "EXAMPLE_COIL": {
            "register": 123,
            "len": 26,
            "val": [1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 0]
        }
    },
    "HREGS": {
        "EXAMPLE_HREG": {
            "register": 93,
            "len": 9,
            "val": [29, 38, 0, 1600, 2150, 5067, 2564, 8450, 3456]
        }
    },
    "ISTS": {
        "EXAMPLE_ISTS": {
            "register": 67,
            "len": 1,
            "val": 0
        }
    },
    "IREGS": {
        "EXAMPLE_IREG": {
            "register": 10,
            "len": 1,
            "val": 60001
        }
    }
}

# alternatively the register definitions can also be loaded from a JSON file
# this is always done if Docker is used for testing purpose in order to keep
# the client registers in sync with the test registers
if IS_DOCKER_MICROPYTHON:
    with open('registers/example.json', 'r') as file:
        register_definitions = json.load(file)

# add callbacks for different Modbus functions
# each register can have a different callback
# coils and holding register support callbacks for set and get
register_definitions['COILS']['EXAMPLE_COIL']['on_set_cb'] = my_coil_set_cb
register_definitions['COILS']['EXAMPLE_COIL']['on_get_cb'] = my_coil_get_cb
register_definitions['HREGS']['EXAMPLE_HREG']['on_set_cb'] = \
    my_holding_register_set_cb
register_definitions['HREGS']['EXAMPLE_HREG']['on_get_cb'] = \
    my_holding_register_get_cb

# discrete inputs and input registers support only get callbacks as they can't
# be set externally
register_definitions['ISTS']['EXAMPLE_ISTS']['on_get_cb'] = \
    my_discrete_inputs_register_get_cb
register_definitions['IREGS']['EXAMPLE_IREG']['on_get_cb'] = \
    my_inputs_register_get_cb

# reset all registers back to their default value with a callback
register_definitions['COILS']['RESET_REGISTER_DATA_COIL']['on_set_cb'] = \
    reset_data_registers_cb

print('Setting up registers ...')
# use the defined values of each register type provided by register_definitions
client.setup_registers(registers=register_definitions)
# alternatively use dummy default values (True for bool regs, 999 otherwise)
# client.setup_registers(registers=register_definitions, use_default_vals=True)
print('Register setup done')

print('Serving as TCP client on {}:{}'.format(local_ip, tcp_port))

while True:
    try:
        result = client.process()
    except KeyboardInterrupt:
        print('KeyboardInterrupt, stopping TCP client...')
        break
    except Exception as e:
        print('Exception during execution: {}'.format(e))

print("Finished providing/accepting data as client")

[brainelectronics]

@GimmickNG as far as I can see from the automated tests umodbus.tcp._send_receive has some issues. May you can check this on your local machine with Docker?

[brainelectronics]

@beyonlo I've published https://test.pypi.org/project/micropython-modbus/2.4.0rc50.dev56/ for you, I initially wanted to wait for the tests to pass successfully, but I see the benefit in having the release out for testing

[GimmickNG]

@GimmickNG as far as I can see from the automated tests umodbus.tcp._send_receive has some issues. May you can check this on your local machine with Docker?

It should be fixed now - was because the examples did not call connect() on the client before running its methods. To maintain backwards compatibility, I made the synchronous TCP class auto-connect on initialize; since the asynchronous version cannot do that though, I've added a note in the constructor's documentation about the difference in behaviour.

[beyonlo]

@beyonlo I've published https://test.pypi.org/project/micropython-modbus/2.4.0rc57.dev56/ for testing. Initial tests passed after fixing the flake8 error locally

@brainelectronics Unfortunately I'm still trying to figure out how to use the register definitions, callbacks, etc on the async version. Coud you please or the @GimmickNG help me with an example for the ModBus TCP Slave?

On this reply I had talked about this subject but I think it was forgotten: " Sorry for my ignorance, but I have no idea how to integrate the register definitions, callbacks, etc to run with the async version. With the sync version there is the client.setup_registers(registers=register_definitions) that call the register_definitions, but how that is called by async version? Bellow there is as well a very simple sync example of tcp_client_example.py (based from the tcp_client_example.py ) that I trying to port to use with async version. Could you please, help me how to port that or point me a example of async version using the register_definitions?

My intention is start testing the ModBus TCP Slave and after ModBus RTU Slave, and finally the Masters RTU and TCP. All in async version :) "

Below is the ModBus TCP Slave example that I'm running without errors (got from the async examples: (async_examples.py), but there is no register definitions.

start_tcp_server.py: (async ModBus TCP Slave)

try:
    import uasyncio as asyncio
except ImportError:
    import asyncio

import os

print(f'MicroPython version: {os.uname()}')
from umodbus import version
print(f'Running ModBus version: {version.__version__}')

from umodbus.asynchronous.tcp import AsyncModbusTCP, AsyncTCP
from umodbus.asynchronous.serial import AsyncModbusRTU, AsyncSerial

async def start_tcp_server(host, port, backlog):
    server = AsyncModbusTCP()
    await server.bind(local_ip=host, local_port=port, max_connections=backlog)
    await server.serve_forever()

def run_tcp_test(host, port, backlog):
    asyncio.run(start_tcp_server(host, port, backlog))

run_tcp_test('192.168.43.143', 502, 2)

Output:

$ mpremote run start_tcp_server.py 
MicroPython version: (sysname='esp32', nodename='esp32', release='1.19.1', version='v1.19.1-966-g05bb26010 on 2023-03-13', machine='ESP32S3 module (spiram) with ESP32S3')
Running ModBus version: 2.4.0-rc57.dev56

[GimmickNG]

@beyonlo Sorry I didn't get back to you earlier, I forgot. Basically, the AsyncModbusTCP (i.e. AsyncTCP) class subclasses AsyncModbus which itself subclasses Modbus (like TCP/ModbusTCP), so whatever methods you would call on the client in the tcp_client_examples.py can also be called on the async examples as well.

In effect, you can replace ModbusTCP with AsyncModbusTCP and only change the processing code (i.e. call serve_forever() instead of while True: process())

I'll modify the examples directory later to add those examples - would probably benefit from refactoring the common bits out into a separate file.

By the way, @brainelectronics I think the files should be renamed if possible - the filename *_client_example.py can confuse people because it's actually a server, not a client according to the Modbus organization's redefinition:

The organization is using "client-server" to describe Modbus communications, characterized by communication between client device (s), which initiates communication and makes requests of server device(s), which process requests and return an appropriate response (or error message)

Likewise, the 'host' in *_host_example.py is actually a client, and not a server.

[GimmickNG]

@beyonlo I have updated the examples/ folder in my branch for the async TCP client/server ("host/client" in the repo for now). Although I haven't tested it yet (since the changes were made online), I think it should work since they share the same common code as the synchronous version. I'll do the same for the RTU version later.

[brainelectronics]

By the way, @brainelectronics I think the files should be renamed if possible - the filename *_client_example.py can confuse people because it's actually a server, not a client according to the Modbus organization's redefinition:

The organization is using "client-server" to describe Modbus communications, characterized by communication between client device (s), which initiates communication and makes requests of server device(s), which process requests and return an appropriate response (or error message)

Likewise, the 'host' in *_host_example.py is actually a client, and not a server.

@GimmickNG I'll do this together with other docs and example improvements after your PR is in

[beyonlo]

@GimmickNG Thanks to improve the examples :partying_face:

The async version stop on the await server.bind(). Details below:

To start the tests, I decide do use for now just the minimal example, without callbacks and so on.

$ cat async_tcp_slave_basic.py

import uasyncio as asyncio

import os
print(f'MicroPython version: {os.uname()}')
from umodbus import version
print(f'Running ModBus version: {version.__version__}')

from rd import register_definitions

from umodbus.asynchronous.tcp import AsyncModbusTCP
async def start_tcp_server(host, port, backlog):
    server = AsyncModbusTCP()
    print('debug 1')
    await server.bind(local_ip=host, local_port=port, max_connections=backlog)
    print('debug 2')
    
    print('Setting up registers ...')
    server.setup_registers(registers=register_definitions)
    print('Register setup done')
    print('Serving as TCP client on {}:{}'.format(local_ip, tcp_port))
    await server.serve_forever()


# create and run task
task = start_tcp_server('192.168.43.143', 502, 2)
asyncio.run(task)

$ cat rd.py

register_definitions = {
    "COILS": {
        "RESET_REGISTER_DATA_COIL": {
            "register": 42,
            "len": 1,
            "val": 1
        },
        "EXAMPLE_COIL": {
            "register": 123,
            "len": 26,
            "val": [1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 0]
        }
    },
    "HREGS": {
        "EXAMPLE_HREG": {
            "register": 93,
            "len": 9,
            "val": [29, 38, 0, 1600, 2150, 5067, 2564, 8450, 3456]
        }
    },
    "ISTS": {
        "EXAMPLE_ISTS": {
            "register": 67,
            "len": 1,
            "val": 1
        }
    },
    "IREGS": {
        "EXAMPLE_IREG": {
            "register": 10,
            "len": 1,
            "val": 60001
        }
    }
}

Output:

$ mpremote run async_tcp_slave_basic.py 
MicroPython version: (sysname='esp32', nodename='esp32', release='1.19.1', version='v1.19.1-966-g05bb26010 on 2023-03-13', machine='ESP32S3 module (spiram) with ESP32S3')
Running ModBus version: 2.4.0-rc57.dev56
debug 1

Look that it stop on debug 1 print.

Below is the same example running in sync mode - works:

$ cat sync_tcp_slave_basic.py

import os
print(f'MicroPython version: {os.uname()}')
from umodbus import version
print(f'Running ModBus version: {version.__version__}')

from rd import register_definitions

from umodbus.tcp import ModbusTCP

tcp_port = 502
local_ip = '192.168.43.143'

client = ModbusTCP()
is_bound = False
is_bound = client.get_bound_status()
if not is_bound:
    client.bind(local_ip=local_ip, local_port=tcp_port)

print('Setting up registers ...')
client.setup_registers(registers=register_definitions)
print('Register setup done')
print('Serving as TCP client on {}:{}'.format(local_ip, tcp_port))
while True:
    try:
        result = client.process()
    except KeyboardInterrupt:
        print('KeyboardInterrupt, stopping TCP client...')
        break
    except Exception as e:
        print('Exception during execution: {}'.format(e))
print("Finished providing/accepting data as client")

Output:

$ mpremote run sync_tcp_slave_basic.py 
MicroPython version: (sysname='esp32', nodename='esp32', release='1.19.1', version='v1.19.1-966-g05bb26010 on 2023-03-13', machine='ESP32S3 module (spiram) with ESP32S3')
Running ModBus version: 2.4.0-rc57.dev56
Setting up registers ...
Register setup done
Serving as TCP client on 192.168.43.143:502

[GimmickNG]

@beyonlo That was rather stupid of me - the bind method ends up calling wait_closed so it's effectively the same as serve_forever. Right, a workaround for the time being would be to remove line 165 and 167 of umodbus.asynchronous.tcp, but I'll have to do some more thorough cleaning later, because it seems like the serve_forever method doesn't actually lead anywhere (it's apparently not defined in any of the _itf classes)

[beyonlo]

@beyonlo That was rather stupid of me - the bind method ends up calling wait_closed so it's effectively the same as serve_forever. Right, a workaround for the time being would be to remove line 165 and 167 of umodbus.asynchronous.tcp

@GimmickNG I did that change as you told me, but another error happens. Bellow the details:

Output:

$ mpremote run async_tcp_slave_basic.py 
MicroPython version: (sysname='esp32', nodename='esp32', release='1.19.1', version='v1.19.1-966-g05bb26010 on 2023-03-13', machine='ESP32S3 module (spiram) with ESP32S3')
Running ModBus version: 2.4.0-rc57.dev56
debug 1
debug 2
Setting up registers ...
Register setup done
Serving as TCP client on 192.168.43.143:502
Traceback (most recent call last):
  File "<stdin>", line 26, in <module>
  File "uasyncio/core.py", line 1, in run
  File "uasyncio/core.py", line 1, in run_until_complete
  File "uasyncio/core.py", line 1, in run_until_complete
  File "<stdin>", line 21, in start_tcp_server
  File "umodbus/asynchronous/tcp.py", line 60, in serve_forever
AttributeError: 'AsyncTCPServer' object has no attribute 'serve_forever'

but I'll have to do some more thorough cleaning later, because it seems like the serve_forever method doesn't actually lead anywhere (it's apparently not defined in any of the _itf classes)

Don't worry, when you fix that, I will restart the tests!

[GimmickNG]

@beyonlo Please try the async tcp examples now, it seems to be working for the most part on my environment. However, I tested it with CPython and not Micropython, so there may be some differences in the implementation. Also, I get some struct errors in the async_tcp_client_example when using it with the async_tcp_host_example, but I'm not sure whether it's because of CPython or not. In any case, I'm not sure whether that's because of the underlying modbus implementation or the async tcp communications themselves - if you also get the same error, then I'll take a look at it probably on Thursday or Friday.