Async Serial Server does not reconnect after losing connection

[peufeu2]

Greetings! I'm having a bit of trouble...

Versions

• Python: 3.11
• OS: Arm linux 5.15.80-sunxi on Orange Pi Lite
• Pymodbus: 3.7.2
• Modbus Hardware (if used): Waveshare USB-RS485
• Server: RTU async

Description

• Start serial async server using USB-RS485 interface, it receives requests and works fine.
• Disconnect USB cable.
• Wait a few seconds.
• Reconnect USB cable.
• Async Server does not resume responding to requests.

Code and Logs

Code snippet:

        self.server = await StartAsyncSerialServer( context=self.server_ctx, 
            framer          = ModbusRtuFramer,
            ignore_missing_slaves = True,
            auto_reconnect = True,
            port            = self.port,
            timeout         = 0.3,      # parameters used by Solis inverter on meter port
            baudrate        = 9600,
            bytesize        = 8,
            parity          = "N",
            stopbits        = 1,
            strict = False,
            )

Log:

[2024-09-19 15:40:42,993] DEBUG:Processing: 0x1 0x3 0x0 0x10 0x0 0x2 0xc5 0xce
2024-09-19 15:40:42,996 DEBUG logging:103 Getting Frame - 0x3 0x0 0x10 0x0 0x2
[2024-09-19 15:40:42,996] DEBUG:Getting Frame - 0x3 0x0 0x10 0x0 0x2
2024-09-19 15:40:42,999 DEBUG logging:103 Factory Request[ReadHoldingRegistersRequest': 3]
[2024-09-19 15:40:42,999] DEBUG:Factory Request[ReadHoldingRegistersRequest': 3]
2024-09-19 15:40:43,001 DEBUG logging:103 Frame advanced, resetting header!!
[2024-09-19 15:40:43,001] DEBUG:Frame advanced, resetting header!!
2024-09-19 15:40:43,005 DEBUG logging:103 validate: fc-[3] address-16: count-2
[2024-09-19 15:40:43,005] DEBUG:validate: fc-[3] address-16: count-2
2024-09-19 15:40:43,008 DEBUG logging:103 send: 0x1 0x3 0x0 0x20 0xf0
[2024-09-19 15:40:43,008] DEBUG:send: 0x1 0x3 0x0 0x20 0xf0
2024-09-19 15:40:43,064 DEBUG logging:103 Connection lost server due to device reports readiness to read but returned no data (device disconnected or multiple access on port?)
[2024-09-19 15:40:43,064] DEBUG:Connection lost server due to device reports readiness to read but returned no data (device disconnected or multiple access on port?)
2024-09-19 15:40:43,067 DEBUG logging:103 Client Disconnection server due to device reports readiness to read but returned no data (device disconnected or multiple access on port?)
[2024-09-19 15:40:43,067] DEBUG:Client Disconnection server due to device reports readiness to read but returned no data (device disconnected or multiple access on port?)

There is no further log, even after reconnecting the USB cable.

StartAsyncSerialServer() function does not exit, so it is still running, but not reconnecting.

Have a nice day!

[janiversen]

Looks like a bug, I will have a look.

[janiversen]

Did you remember to restart your client ? depending on which client you use it does not have automatic reconnect

The server does not do reconnect, it waits for a new connection..connections are always initiated by the client,

[peufeu2]

Yes indeed the client was sending requests, the "RXD" LED on the USB-RS485 interface was blinking all the time, but no "TXD" since the server was frozen and did not reply.

The client is a solar inverter, and the server is a pymodbus smartmeter emulator. The inverter keeps querying the meter rather fast, something like 10 times/s. If the server does not reply, the inverter keeps banging it with requests continuously until it does. Basically, it wants smartmeter data in real time as fast as it can get it. In fact this could be the source of the issue, maybe the server has trouble restarting while receiving too many requests? But then I guess it would complain in the logs, and the log is silent after it dies.

[janiversen]

something is wrong here, a solar inverter is a server not a client!

The smart meter is normally also a server, but it normally do not query the solar inverter but merely measures the load on the line.

A client ask for data, and a server produces data. A server never asks for data.

[janiversen]

Wait a sec, I get the setup.....the pymodbus server measures the load on the voltage lines and report it back.

[janiversen]

The pymodbus server, should reopen the com port after a disconnect, that might be the problem I will investigate but not before monday.

[peufeu2]

Yes, the smartmeter measures power drawn by the house from the grid, and the inverter queries it to know how much it should produce. So in this case the inverter is the client, and the smartmeter is the server. The inverter actually has two RS485 ports: one for the smartmeter on which it acts as client/master, and another for control on which it acts as a server/slave, that's what you were thinking about I guess. That's pretty standard for solar inverters these days.

My use case is controlling two inverters that don't want to work together. I can't connect them both to the same meter, because they both want to be master on the meter bus. I can't add another meter, because both meters would read the same power, so each inverter would output that power, and total output power would be double what's needed.

So I inserted this python code (running on a Pi) in the middle:

On one RS485 port, it acts as client and queries the real smartmeter.

On 2 other RS485 ports, it acts as servers (one per port), pretends to be a smartmeter (one per inverter), and my 2 inverters query it. It feeds them tweaked measurements to split the power demand between the two inverters.

It all works perfectly, I'm charging my car with it. I mean pymodbus really did save the day to the tune of about 4000€, as the guy who sold these inverters to me promised they would work fine in parallel, but it turned out they don't... well now they do!

[janiversen]

It is confirmed that the serial port gets closed and do not reopen, a fix will soon be available.