iot-data: shadow state delta is backward

[rtandy]

Hi, thank you for providing this very useful library.

I found a few issues in the implementation of IoT device shadow deltas:

1. It looks like moto computes delta as the difference from desired to reported. This is backward; it should be the difference from reported to desired.
2. Keys that are present in reported but not desired should not be included in delta. If desired is empty or not present, delta should not be present at all.
3. While #3850 fixed deleting keys by setting them to null, it still saves null as the value if the key was not already present.
4. Jsondiff symbols such as replace are not JSON serializable and cause a TypeError.

Let me know if I should create separate issues for some or all of these. Thank you for working on moto!

Test cases:

1: (key present in both reported and desired, values differ)

Input:

{
  "state": {
    "reported": {
      "online": false
    },
    "desired": {
      "online": true
    }
  }
}

AWS response:

{
  "state": {
    "desired": {
      "online": true
    },
    "reported": {
      "online": false
    },
    "delta": {
      "online": true
    }
  }
}

Moto response:

{
  "state": {
    "desired": {
      "online": true
    },
    "reported": {
      "online": false
    },
    "delta": {
      "online": false
    }
  }
}

2: key present in reported, absent from desired

Input:

{
  "state": {
    "reported": {
      "online": false,
      "enabled": true
    },
    "desired": {
      "enabled": true
    }
  }
}

AWS response:

{
  "state": {
    "desired": {
      "enabled": true
    },
    "reported": {
      "online": false,
      "enabled": true
    }
  }
}

Moto response:

{
  "state": {
    "desired": {
      "enabled": true
    },
    "reported": {
      "online": false,
      "enabled": true
    },
    "delta": {
      "online": false
    }
  }
}

2a: desired absent entirely

Input:

{
  "state": {
    "reported": {
      "online": false
    }
  }
}

AWS response:

{
  "state": {
    "reported": {
      "online": false
    }
  }
}

Moto response:

{
  "state": {
    "reported": {
      "online": false
    },
    "delta": {
      "online": false
    }
  }
}

3: setting a previously absent key to null

Input:

{
  "state": {
    "reported": {
      "online": null
    }
  }
}

AWS response:

{
  "state": {

  }
}

Moto response:

{
  "state": {
    "reported": {
      "online": null
    },
    "delta": {
      "online": null
    }
  }
}

4: TypeError when one or the other (but not both) of reported and desired is an empty map

Input:

{
  "state": {
    "reported": {
      "online": false
    },
    "desired": {}
  }
}

AWS response:

{
  "state": {
    "reported": {
      "online": false
    }
  }
}

Moto response:

TypeError: keys must be str, int, float, bool or None, not Symbol

[bblommers]

Hi @rtandy! Thank you for raising this and for providing the examples of where we go wrong!

I've raised a PR that fixes all these, and added tests that also validate the behaviour against AWS - so I believe we should have parity when that is merged.