Best method for handling reconnections to large numbers of PVs

[kathryn-baker]

Hello,

In our group we are using pvapy for a number of applications but primarily to host a number of PvaServers. We then have a separate application that connects to the PVs on these servers using the Channel client to monitor them for changes.

So far, the functionality of both applications has worked well, but we are now facing a number of issues with reconnection protocols. I have tried two different methods to account for PV reconnections so far and both have created their own problems. I'm not sure if these are a result of how I'm doing the reconnection using the library or something in our system...

The methods I've tested so far are:

1. I create the Channel object for the client, if the PV is down and it times out, we subscribe the onChanges function to monitor for updates anyway and just wait for the channel to connect and start receiving messages at some point in the future

This method works well if the PvaServer is stopped cleanly, but encounters problems if the PvaServer crashes unexpectedly. In this case we stop receiving updates for the PVs through the assigned onChanges function, even though we can see the PVs updating with a pvmonitor on the command line.

3. I create the Channel object for the client and assign a connection callback using the setConnectionCallback() method. In this method we subscribe the onChanges function and start monitoring the PV. When it disconnects, we stop monitoring the channel.

This second option feels to me to be the cleaner of the two. However, we noticed alternating crashes between the PvaServer and the client application. We haven't yet worked out which of the two programs is responsible for the crashes. We also encountered some segmentation faults using this method.

In your opinion, what is the best way to handle these situations where potentially lots of PVs become disconnected at once? Do you have any advice for how to handle reconnections on the client side, and is there anything you might be able to suggest for the server side to improve the closure of PV connections?

Thanks for your help!