New integration: TED (The Energy Detective)

[rianadon]

Breaking change

Proposed change

Adds a new integration TED that supports both the TED5000 and TED6000 energy monitors. These are both popular energy monitoring solutions, so the integration will help users looking to integrate these energy monitors into Home Assistant's energy panel.

This integration differs from #54555 in that the old TED5000 integration is kept so that there are no breaking changes with the pull request.

Users using the old TED5000 integration will see the following in their repairs panel:

The features are as follows:

• Shows MTU breakdown of power and voltage (same as the ted5000 integration)
• NEW: Shows today's and month-to-date totals of MTU usage (which can be integrated with energy panel)
• NEW: Shows system-wide totals of power usage, power returned to grid, and power consumed from grid
• NEW: (For TED6000) Shows Spyder breakdown of current, today, and month-to-date total power usage
• NEW: Configuration options to create entities for MTU breakdowns and Spyder breakdowns

Type of change

• [ ] Dependency upgrade
• [ ] Bugfix (non-breaking change which fixes an issue)
• [x] New integration (thank you!)
• [ ] New feature (which adds functionality to an existing integration)
• [ ] Deprecation (breaking change to happen in the future)
• [ ] Breaking change (fix/feature causing existing functionality to break)
• [ ] Code quality improvements to existing code or addition of tests

Additional information

Brands pull request: https://github.com/home-assistant/brands/pull/3897 Documentation pull request: https://github.com/home-assistant/home-assistant.io/pull/19234

• This PR fixes or closes issue: fixes #
• This PR is related to issue:
• Link to documentation pull request:

Checklist

• [x] The code change is tested and works locally.
• [x] Local tests pass. Your PR cannot be merged unless tests pass
• [x] There is no commented out code in this PR.
• [x] I have followed the development checklist
• [x] The code has been formatted using Black (black --fast homeassistant tests)
• [x] Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

• [x] Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

• [x] The manifest file has all fields filled out correctly.
  Updated and included derived files by running: python3 -m script.hassfest.
• [x] New or updated dependencies have been added to requirements_all.txt.
  Updated by running python3 -m script.gen_requirements_all.
• [ ] For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
• [x] Untested files have been added to .coveragerc.

To help with the load of incoming pull requests:

• [ ] I have reviewed two other open pull requests in this repository.

[natethelen]

I haven't been able to test it in this branch yet, but when I copied this ted folder to my dev branch and start the wizard, I get an error that there are conflicting dependencies in tedpy

[rianadon]

Indeed. It looks like the version of httpx in tedpy needs to be updated.

[natethelen]

BTW, I have a fairly large setup with 4 MTUs and 2 spiders on 2, 200 amp main panels, solar, and a 100 amp sub panel. I am running into some problems and will debug in the morning.

[realumhelp]

BTW, I have a fairly large setup with 4 MTUs and 2 spiders on 2, 200 amp main panels, solar, and a 100 amp sub panel. I am running into some problems and will debug in the morning.

Some of the Solar stuff we kinda honestly winged since we didn't have the equipment. Did you by chance test the underlying software library standalone to make sure it is pulling the correct data?

https://github.com/rianadon/the-energy-detective-py

I haven't toyed with it for a while but it might be a bit of fun to get back into it!

Edit: Winged is not the right word we followed the spec but some of the configurations can get quite complex and we had no way of testing them fully.

[natethelen]

Some of the Solar stuff we kinda honestly winged since we didn't have the equipment. Did you by chance test the underlying software library standalone to make sure it is pulling the correct data?

I have used tedpy and everything looks good when using it directly.

The error I am getting is in TedSensor.native_value for the sensor with description of "net_now". In this case self.coordinator.data.get(key) returns None and an exception happens when trying to getattr of None. Still digging, but if that sparks something in your brain...

[natethelen]

The error I am getting is in TedSensor.native_value for the sensor with description of "net_now". In this case self.coordinator.data.get(key) returns None and an exception happens when trying to getattr of None

It seems that the coordinator has the "net" values named "energy". I will do a quick change to see if that solves the problem.

[natethelen]

It seems that the coordinator has the "net" values named "energy". I will do a quick change to see if that solves the problem.

Changing line 45 of ted/init.py to this solves my error:

data["net"] = ted_reader.energy()

Now just waiting to see if the energy graphs look correct...

[rianadon]

Thank you! I took a look again and this seems like the correct fix. I was deciding between giving the net energy sensors ids with net or energy and chose both I guess. Now they're all named net.

[rianadon]

I also noticed this PR: https://github.com/home-assistant/core/pull/57933. It looks like it adds a few other sensors we don't have in this PR like the number of days left in the billing cycle.

Perhaps it would be good to add those sensors once this PR is merged.

[natethelen]

I few other things I noticed:

• When there is solar as part of the installation the "net" energy (both the totals and the MTUs) can be negative. Consequently, the state_class for the daily and mtd sensors cannot be TOTAL_INCREASING and must be TOTAL.
• In the _now values in the SensorEntityDescription you could use device_class=SensorDeviceClass.POWER
• POWER_WATT is deprecated (replaced with UnitOfPower.WATT)
• ENERGY_WATT_HOUR is deprecated (replaced with UnitOfEnergy.WATT_HOUR)

Will continue to test and review...

[natethelen]

Naming Note: In a system that has Solar, the .consumption() values are how much total energy used in the house/office/etc, not necessarily the amount "consumed" from the utility. In this case, .energy(). is the amount consumed from (or returned to, if negative) the utility. It may be less confusing to use stype of "Energy Usage" for "consumption" and "Energy Consumption" for "net". Thoughts?

[natethelen]

I am seeing a good number of these: "Timeout fetching TED TED 6000 data", then recovering the next try. Maybe with a larger deployment like mine it regularly takes close to 10 seconds. Bump timeout up to 15 seconds?

[rianadon]

Thank you for all the detailed feedback and suggestions!

I agree with using "Energy Usage" for "consumption", but I find "Energy Consumption" still too confusing for the amount consumed from the grid. I think either "Grid Energy Consumption" or "Net Grid Energy" (what I'm now using in the code) would be more clear.

[natethelen]

Looks great. Ship it! :)

[natethelen]

FYI, I have been running this pretty consistently for a few days without issue

[Quentame]

@natethelen : Anyone can review the code of an HA PR using comment/review actions on the "Files changed" tab 😉, not only members of home-assistant/developers !

We actualy encourage people to do so ☺️ !

And using those tools (code comment/review) improve following history of the PR.

[SupremeSports]

Could you integrate the utility sensors such as actual kWh cost, Time Of Use, Plan type, Carbon Rate, etc. I already did it in my now closed PR#57933

[rianadon]

@Quentame Thank you so much for the detailed PR review! I really appreciate all the feedback. I've addressed your comments in the recent commits.

@SupremeSports I have my hands a bit full this week. I'm thinking these sensors would be great to add in a second PR?

[SupremeSports]

@SupremeSports I have my hands a bit full this week. I'm thinking these sensors would be great to add in a second PR?

As per my experience (with my PR) it can be long. Mine took over 15 months and was still not processed. I'd suggest to put everything there all at once...

[Quentame]

@SupremeSports I have my hands a bit full this week. I'm thinking these sensors would be great to add in a second PR?

As per my experience (with my PR) it can be long. Mine took over 15 months and was still not processed. I'd suggest to put everything there all at once...

I'm sorry for the looooonnnng time your PR was ... not finalized, it happened to me ... it happens to everyone, including the best 😅 ! As #57933 seems to complexify, and as I already statred reviewing this code, I would prefer in a separate PR too (sorry), either you or @rianadon would create it. It will avoid me to read everything again and again.

Ping me on that PR, I'll do my best fast reviewing 😌 Stay strong 💪

We will also ask davet2001 as it is the code owner of ted5000, and starts reviewing your old PR, he could also help.

[davet2001]

As per my experience (with my PR) it can be long. Mine took over 15 months and was still not processed. I'd suggest to put everything there all at once...

My suggestion would be to make a PR as small as possible. The short ones are closed quickly.

[rianadon]

Thank you for the review @davet2001. I think adding tests for the other components in a separate PR would be a good idea.

Also, the TED config flows are fully covered by the tests. I'm not sure why the code coverage reports that the PR does not meet the coverage % requirements.

[rianadon]

@davet2001 @Quentame this is ready for another review.

[davet2001]

@rianadon Yes, a function is probably only necessary if there's a need for it to change based on parameters. Could use a constant if it's needed in more than one place.

I think a new constant deprecation has caused the test to fail, please could you take a look?

[rianadon]

Thank you! I've updated the constant.

[rianadon]

@Quentame I see you started a review. Would you mind providing that 2nd review/opinion?

[rianadon]

If there's anyone on the thread who would like to provide a second review, I'd really appreciate it.

It's been almost a month since the last review, and it would be great to have support for all TED devices on Home Assistant.

[rianadon]

Hi @emontnemery. As I mentioned before I had held off implementing the state class for net meters because I was a bit confused how to proceed. Your comment helped clarify things, but I'm still facing an underlying problem caused by the Ted 6000 device resetting at inconsistent times.

I had asked on the discord and got no response, so I figure I'll ask here.

I've been running my changes with last_reset for a few weeks, and today I decided to create a statistics card for one for one of the net meters to check how they're working.

This is what I got. Based on the form of the state measurement, the sum should be an ever-increasing line. However, it has jaggies because the time that the sensor resets is inconsistent. Sometimes it's at midnight, but other days it can be as far as five minutes behind.

As comparison, here's a non-net energy value that uses the total_increasing state class. Notice how the line is straight.

Is there any great way to handle the inconsistent reset times? And if not, I'm thinking that maybe I should not give these net entities a state class measurement. The Ted6000 also publishes historical data (for example what was the total net energy use 6 days ago). Perhaps I could use these to detect whether the day has rolled over, but it sounds like a lot of extra logic. Perhaps this is better left to a future PR? It would be great to get this work merged.

[emontnemery]

it has jaggies because the time that the sensor resets is inconsistent. Sometimes it's at midnight, but other days it can be as far as five minutes behind.

The problem is that you're trying to calculate the last_reset time based on the local time. That simply won't work, you need to either get the time from the API or detect that a new period has started and bump the last_reset only then. An example of the former is if the API includes some meta data indicating for which date the reading is.

Let's chat on discord if the way forward is not clear, I'm emontnemery#6618 there

[github-actions[bot]]

There hasn't been any activity on this pull request recently. This pull request has been automatically marked as stale because of that and will be closed if no further activity occurs within 7 days. Thank you for your contributions.