nodejs-poolController icon indicating copy to clipboard operation
nodejs-poolController copied to clipboard

Documenting Pentair Intellivalve communications

Open jeffegg opened this issue 2 years ago • 111 comments

Describe the bug Not really a bug, more of a feature enhancement. And maybe this isn't the correct place for this, maybe the wiki would be better. Let me know Currently (as I understand) the intellivalve is minimally supported in nodejs-PoolController. In my copious extra time, between family and work, I decided to see what I can do to help improve support. So I ordered an extra intellivalve when adding some to my pool in hopes that I can help this out. Seems I am able, looking at the hardware, there is a PIC16F1718 inside (Yay), as well as a ICSP port, so using MPLABX and an ICD4, I dumped the FW. I'm currently in the process of dissembling the FW from using MPLAB, and Ghidra (thanks NSA :)) I don't know what will be supported, but since I actually made the valve do something from message Center based on the FW (reset) maybe I can figure out something useful, like setting new endstops, etc. Else, I've reverse engineered a partial schematic for the valve and I'll write my own FW. Both efforts will probably take a few weeks, depending on work schedule. FYI I use Pool controller in Nixie single body mode, and I don't have a Pentair panel to reverse engineer communications (assuming the panels have some interesting support.) This would be easier to capture packets and reverse engineer, but I'm not interested in the panel, so don't want to spend the money on it.

Commands I know (as of 10/31/21)

SW reset of valve: Action: 0xF7 (247)

Sent Packet: FF00FF A5 3F 0C 10 F7 04 01 01 F9

 Dissasmbled FW:           
          if (((RX_BUFF_OTHER?[7] == 0xf7) && (RX_BUFF_OTHER?[5] == DAT_DATA_006b)) && (RX_BUFF_OTHER?[9] == 1)) {
                      FUN_CODE_32aa(1); // Not sure what this does yet
                      reset();
_DAT_DATA_006b // Looks to be the valve address, I'm guessing with multiple valves we probably can address, mine is set to 0xC(12) by default._ 

Response Packet: Source: 12 (Valve); Dest: 16(OCP), Action: 1, Data Length 1: 247, Header [165,1,16,12,1,1] Term: [0,167]

jeffegg avatar Nov 01 '21 05:11 jeffegg

Hi and welcome. Interestingly enough, there are no Pentair outdoor control panels (OCP's) that communicate with the Intellivalve via RS485 yet. The valves work exactly like the Jandy valves in that they operate on a +/- DC voltage. The valves are more accurate (less drift over time) but for whatever reason Pentair has not released this functionality to the world yet. @rstrouse started a project over 1 year ago, valve-Grooter, to try and brute force it. There are a number of packets (245, 247, 240/241) that do things but we were never able to put them together in any sequence that would set or move the valves. We did brick a number of them and after sending 10's of millions of messages we eventually gave up. If you continue to find more in the FW, we can triangulate, but it will take someone with your skills or Pentair to help un-stuck us from our progress so far.

Check out the repo above and see if it matches up with your findings and/or if it helps you figure anything else out.

tagyoureit avatar Nov 01 '21 05:11 tagyoureit

Thanks! I knew there was an effort before to brute force it, saw it on some forum. But I was unable to re google it. I just started on the 240 command :). Good news is I have the original FW saved, before even hooking up the RS485, so if I brick something I should be able to recover assuming I don't wear out the flash (doubtful). I could remove the unit if I did this, but this part is on a 6 month to 1 year lead time it seems. Thanks for the link will see what I can do.

jeffegg avatar Nov 01 '21 05:11 jeffegg

Good luck. I am not so sure that flash isn't the failure point. Maybe we hit something that trashes the firmware but it didn't seem that a byte sequence sent the valve south. It appeared to be based on how long the valve was having 245 and 247 messages written, We never did see anything that would tell us what the current status of the valve is.

Btw the voltage on the pins is 24vac not dc.

rstrouse avatar Nov 01 '21 15:11 rstrouse

Hmm, well thankfully this valve is dedicated to science, worst case I have an emulator I can solder in till I get some new PICs. I can definitely see places where buffer overflow could overwrite something important.

Commands I can see in the source code: Read Destination = 0xC(12) with a command of 0xf0 (240) This returns a serial number (read from the flash chip and assuming it is unique) plus some other fixed and variable data Valve returns command 0xF1 and an 18 byte payload (here is what mine read): 0080 801F124FD63E 0102FFFF 305B2001 0040 0080 is constant 801F124FD63E look to be the serial number read from flash at valve poweron or reset, also the periodic hail has this 0102FFFF is constant 305B2001 these are variables, but I don't see where these come from yet. 0040 is constant

0x50 (80) This command seems to set the Valves Address. After the valve address is set, we no longer see the hails (at least I think that is what they are.)

Header [165, 63,12,15,80,8] It is important to have the source as broadcast (15 or something else), when it is set to 16 (Panel) the address is reset (not sure if this is because I'm missing something else or just the way it is

Payload [ADDRESS, *(don't care it seems), serial number from hail or the reading 0xF0] so in my case I set the valve address to 0 with 0x801F124FD63E as my serial number [4,0,128,31,18,79,214,62]

Upon sending that I get back a broadcast packet from the valve with Source as address above (0x4) destination Panel(16), Action of 1, and payload of 80

image

0x51 (81) I don't know what this does, but it only works when the valve address is 0xC. I haven't really dug into it yet.

I'm still not seeing output when I do anything to the valve, but I suspect there is something else I need to set, will take some more time to continue to disassemble

jeffegg avatar Nov 05 '21 08:11 jeffegg

Still haven't done anything useful, but wanted to document this in case anyone tries. There are some commands that can be run at valve startup when the mode light is blinking blue: If you write an action of 245 (0xF5) with payload bytes 2 or bytes 3 not equal to 0, the valve is put into some odd mode (only sends the Hail command, keeps resetting, and buttons don't work.) This command writes the eeprom address 0x41 to 0xFE. Not sure what this command is for yet, but if you see the mode lights blinking blue and valve is useless, you can send something like below to recover (the rest of the payload may or may not matter, mainly sending bytes 2 and 3, to change the EEPROM address 0x41 0xFD, and get out of the odd mode image

jeffegg avatar Nov 07 '21 09:11 jeffegg

Yes you have found the blinky blue mode. Originally I thought this might be the valve waiting for input from the master but never could get anything else out of it. I have a one valve stuck in blinky blue and another in modified watermelon mode. I'll see if I can break those out of their respective mode this week using the command above.

Here is some background on our results.

Action 82

This message is referred to as "I am Groot" simply because when we were initially sending messages to the valve this is the only response we got from the valve. It is broadcast from the valve periodically, regardless of whether it is asked for by the master. So it is not a response to a request from the RS485 master. There is one other device that does this and it is iChlor. Both of these appear to fire this off to address 12.

All groot message payloads are unique to the valve and there are similarities for the first several bytes of the payload if the valve is manufactured around the same time. We spent quite a bit of effort to try making sense of this output by comparing it to the label on the valve. Nothing meaningful was gleaned from that other than it looked like you could determine whether one valve was older or younger than another and byte(5) was sequential for valves purchased about the same time.

Action 80

Action 80 appears to set the address by sending the desired address in the first byte followed by the valve key. The valve key is the last 6 bytes of the groot payload. Upon successfully setting the address the valve will respond with an ACK (action 1) payload 80 from the address you assigned.

Action 240

This is an interesting message. When you send an action 240 to the valve using the address it will respond with a 241 message that contains the unique valve key followed by an additional 10 bytes tacked to the end of it which was identical for all valves found in the wild.

So if you have multiple valves on the bus and send an action 240 to 12 then all valves that have not been addressed will respond with a 241 containing the unique key and some sort of status bytes. If the valve loses power and is not stuck in a reboot cycle from 245 or 247 then it will revert to address 12. Address 12 appears to be a general broadcast address for several Pentair equipment items in the wild.

Action 241

Action 241 is a response message. This is a message that is sent from the valve when requested by an action 240. It never shows up unless it is asked for (like a good RS485 citizen). It also honors the proper addressing once you set an address with action 80. The payload for this message has been identical for all valves in the wild and has not changed unless the valve is in a mode where it no longer responds to the buttons on the valve.

This payload is in the form with constant data between all valves witnessed as follows [0, 0, <6 bytes for the valve key>, 1, 2, 255, 255, 48, 91, 32, 1, 0, 64]. This is true unless the valve is stuck in blinky blue then the payload changes to [0, 0, <6 bytes for the valve key>, 255, 255, 1, 0, 48, 91, 32, 1, 0, 64].

Action 247

Action 247 will cause the valve to start the blue light on the valve blinking (blinky blue). When this message is sent to the addressed valve with a payload staring with [1, 2, ...] it will respond with an ACK[247].

If you send enough of these to the valve then it will stay that way and continue rebooting. The human interface on the valve will no longer respond but the valve will continue to respond to messages. Like mom used to say if you keep making that face it will be stuck that way. That is what happens after you have beat the valve up with 247s. However, one of the tests I am going to perform is to send the 245 message again to see if I can break free from the blinky blue on one of the valves stuck there.

Action 245

Action 245 seems to create light shows on the red segment leds for the valve positions. By sending different combinations in the payload we seem to be able to get different lights illuminated on the valve. This message responds with an ACK[245] when a combination it seems to like is sent.

For instance, what was identified as watermelon mode could be attained by sending [0, x, x, x, x, x, x, x, x, x]. Beating up the valve with these messages eventually gets you into carrot mode. Where it doesn't carrot all what you send it and it doesn't carrot all which buttons you press on the valve.

We also hooked the valve up to a relay to cycle the valve after different responses, sending action 240 to see if there was a different response (241) from the valve but never got beyond the aforementioned byte changes and the position never seemed to report anywhere.

In the end we gave up after several valves in the wild stopped responding to the buttons on the valve but never got to the point where the RS485 responses stopped. The only thing I can surmise from it is that the valve either is gets stuck with bad data on it or we simply wore out the flash by writing several hundred million messages to the same locations. There has to be some persistent storage on this valve or it would not remember endpoints or the current mode selected when power is lost.

Bear in mind we hammered this with a brute force method since the firmware code was not available from the valve itself.

rstrouse avatar Nov 09 '21 17:11 rstrouse

"There has to be some persistent storage on this valve or it would not remember endpoints or the current mode selected when power is lost." There is persistent storage in 2 places:

  1. I only see this being read, never written to, but still disassembling (this will take some time, not friendly to go from hex to pic assembly to c code without context. There are a few spots in the code where values are pulled from the code binary. I'm guessing these are some const values.
  2. There is a 5 pin i2c eeprom onboard I missed it at first, I was expecting 6 pins and mistook it for something like a linear voltage regulator. I cannot decode the package marking, but soldering down some probes and hooking up an oscilloscope, plus what I see from the dissembled code this seems to be some 24xxx eeprom. I need to pull this image in case I mess something.

On a good note I was able to get some rs485 traffic to emit when the valve changes position, I can only do this by writing a data value (inside the pic) when in a debugger and the output on the bus is missing the standard 0xff 0x0 0xff message header. Also good news is that each step and direction seem to have different values. I'm trying to trace the writing for this variable but not seeing this, yet. I think the dissembler is messing up bank selects somewhere but will take some time to trace. It is also possible that this is some type of manufacturing mode for ADC calibration (the valve uses a pot to set the position).

Blinky blue mode reminds me of a firmware update mode, but I haven't found proof of that yet. It could also be a mode where the rs485 gets exclusive access to the valve...

Also to note there are 2 different rs485 handlers (I call them early handler and main handler) and they are a bit different from each other. When the valve is powered on, or reset the early handler is used it hands off to the main handler after the valve blinks blue a few times and before the red position and green mode light come on. Based on this I'm wondering if there is some flow expected by the valve for a handshake:

  1. Look for the hails from the valve (action 82)
  2. Set valve address (action 80)
  3. Reset the valve (action 247 with payload = 1)
  4. listen for hail on reboot(action 82)
  5. reconfigure valve address (action 80)
  6. send action 245 mode to put into remote/rs485 exclusive mode. This action only seems to be available in the early handler, but again could be missing somethibg
  7. then send some commands to program valve.
  8. move to the main flow with updated and saved info. I'm really speculating here, I don't know the normal pentair handshakes, etc. Was trying to think how I would do it if I were to rewrite the firmware

jeffegg avatar Nov 09 '21 18:11 jeffegg

It is interesting that if you send a 240 out to address 12 all valves will respond with a 241. However, if you have a bit of RS485 understanding the notion of multiple slaves responding to a single master message is very messy and unreliable in that every valve responding at once can destroy the response from all of them. I suspect the default address is simply set to 12 and it blindly responds when it sees that address.

Since there is no selector to identify a valve address, perhaps our groot message (82) is the only viable option. So it is likely that the OCP will take however long to listen for groot messages on the bus for all unassigned valves addresses but only after it has assigned the ones already defined in the current definition. The valve does appear to inspect all 80 messages for the 6 byte key to determine if it is the intended recipient. The only way to get this key is by listening for the valve to broadcast it. To verify whether the address has been properly assigned you can send a 240 to the assigned address and it should return the valve key.

I suspect that there is a specific address range that Pentair has in mind for these with the most likely address range that starts at 160 and ends at 180. This puts the equipment identifiers outside of other equipment on the bus and provides a contiguous 20 address range that covers all potential valves available with 3 power centers. It could go as high as 24 with reserved addresses for specific valves such as intake, return, and heater bypass.

So a likely sequence would be set all the addresses already assigned by sending action 80 on OCP boot verifying each one by calling 240 for verification using the newly assigned address. This 241 result will match the valve key to the assigned address. The address is critical since the address can only a single byte and the valve key is 6 bytes. This is also critical for the OCP configuration since the valve configuration on the OCP needs to map to a single byte ordinal on one of the 15 valve messages output by the OCP. This too is speculation but it does match up to how Pentair addresses and manages other equipment on the RS485 bus.

Rumors are that there will be 5 slots in the Pentair programming so the valve would have 5 potential endpoint pairs. What is weird is how they would expect to engage these given the current 1 to 1 nature of the assignments where 1 circuit is assigned to each valve. If the circuit/feature is on then the valve is moved to the diverted position. This is done by simply applying power to either the red or white wire via an SPDT relay. The motor simply rotates until the valve reaches the assigned endpoint.

So it is assumed that turning on a circuit or feature will look at all running features associated to the valve and perhaps re-program the endpoints on the valve before engaging it using the relay. How it determines which endpoint slot to use is a mystery. As I look at the operation of the valve the endpoints seem to come in pairs but the only thing that would seem to matter would be the diverted position on the valve. No matter how you slice it if all the associated circuits are off then the un-diverted position would always be the same.

To make this really useful you would want a scenario where the valve endpoint is dependent upon multiple circuits being engaged. For instance, if the waterfall and the fountain are on divert more water to the feature plumbing. I could see a scenario where the max diverted position is calculated by identifying all circuits/features that are currently on and picking the largest diverted position value. This is in line with the way VS/VF pumps work.

rstrouse avatar Nov 09 '21 20:11 rstrouse

Documenting a bit more on action 241 (response to 240) in normal mode: Here is what is written by the FW, I have figured out all the info there. My valve returns the following in hex: 0080 801F124FD63E 0102FFFF 305B2001 0040 0080 is constant in code 801F124FD63E look to be the unique ID of the valve read from flash at valve poweron or reset 0102FFFF is constant in code 305B This is the Device ID of the PIC chip (305B is a PIC16F1718 for reference) 2001 This is the Device Revision ID of the PIC chip 0040 is constant in code

Slow going trying to disassemble and fixup code, and try to infer behavior. Unfortunately not seeing anything in code for control of the valve, but I still have about 60% of the code as unknow behavior. Best I was able to do was to get some RS485 traffic to be sent when changing valve locations and modes, etc. I suspect this is a factory test mode, it can be entered by pressing and holding the Mode and Red button through a power on. Then you need to send a certain RS485 packet to enter the mode. After that the valve basically operates as normal, but sends out RS485 messages on valve change (though buttons) but they dont have the typical Pentair header (255 00 255). Still working on it, but dont hold your breaths this is going to take awhile.

jeffegg avatar Nov 20 '21 06:11 jeffegg

DANGER DANGER DANGER!!!! For those playing around with sending codes to the valve Blinky blue mode entered by sending action 245 with Bytes 0 and 1 non zero is a Firmware update mode. FW update packet looks like: DATA (values are in HEX): 01 02 - What I choose to enter blinky mode 2A 00 - This writes the microcontroller program memory at 0x1500 (PIC use a fixed 16 bit opcode, so 0x2A00 / 2 => 0x1500) xx xx - Not sure if these matter yet, I wrote 0x1 0x2 and didnt see anything happen, I suspect these are just "filler" Bytes after this are written to memory, will use 0 if these bytes are not sent, expecting 64 bytes in total

Please note PIC architecture enforces a whole row (all 64 bytes) to be cleared, so if there is an unalinged write, bytes not written will fill with 0xFF3F (default memory values in erased memory)

I think there is some protection in FW update mode to prevent some overwrite memory I'm guessing for the boot loader. Code for this protection is a bit confusing, trying to understand.

So if the boot loader is protected, we should be able to recover bad valves through the RS485 port. If someone is interested I can probably write a small program to reflash the firmware(FW).

Also this means another option that can be discussed is to write a custom FW for njsPC to control the valves, if I continue to find the valve doesn't allow SW control. It is possible Pentair didn't finish the code and decided that if they want this control later they can upload a new FW.

jeffegg avatar Nov 20 '21 09:11 jeffegg

Well that certainly jives with our experiences. I have 2 valves that are inop after beating it with 245s.

rstrouse avatar Nov 20 '21 22:11 rstrouse

Got stalled on this update with my day job (that oddly is also a night job as well most days) and family. I don't think Pentair finished the code for remotely controlling the value (doesn't mean it doesn't exist and I'm missing it.) I've gone through the FW a few times, and there is a basis there for remote control but missing some of the actual code to send and receive the data. I'm moving towards writing my own firmware(FW), there is a bootloader on the valves to do FW updates via the RS485 lines, and I have some rough python code here: https://github.com/jeffegg/IntelliValveFWUpdater to do updates. If people need to revive their dead valves this could potentially help. Unfortunately I'll have to rewrite most of the code for the valve, I'm considering to see if I can just hack in the support with what is there. But I think this will restrict the implementation too much; but still in the back of my head

I'll probably stage the features out like this:

  1. Get basic "remote" access using red and white wires. (Full swing, i.e. 1 side open the other closed); buttons on valve would not work
  2. Report changes in valve status, and ability to poll status
  3. Remote access via RS485 to override red/white wire selections (still full swing)
  4. Support "set mode" to allow setting non full swing via buttons (both relay driven and RS485 would respect these points)
  5. Support more of Intellivalve feature set
  6. ...

If people are interested in this alternative, I can open to the community on what features, how things are implemented, etc. And to obviously share.

jeffegg avatar Jun 29 '22 08:06 jeffegg

If you could make this happen, there would quite a few happy folks out there. I have two manual valves right now that I would never bother to replace with a traditional JVA24 but would do it in a heartbeat with a remotely controlled, variable diverter. One is my spa bypass and the other is the diverter between the floor returns and the wall returns.

My ideal would be to remotely set a fully open/fully closed position. And then being able to send it commands to open a certain amount. Doing this by absolute value would be good (maybe degrees, maybe abs value depending on what you find). Everything that operates based on % just seems hokey (chlorinators, hayward pumps). I should be able to send an RS485 command and have the return diverter fully open (floor returns) when my heater is on full blast, fully closed when my cleaner is running, and somewhere in the middle for just the filter pump running.

tagyoureit avatar Jun 29 '22 15:06 tagyoureit

This would be huge - as @Tag said, not having remote control is what's keeping me from replacing all my valves with iValves

johnny2678 avatar Jun 29 '22 22:06 johnny2678

I have 8 IntelliValves currently installed. Happy to help where I can.

rstrouse avatar Jul 05 '22 20:07 rstrouse