core icon indicating copy to clipboard operation
core copied to clipboard

Avoid joining states_meta for statistics queries

Open bdraco opened this issue 1 year ago • 2 comments

Needs #89929 first

Proposed change

This is an additional followup to improve the situation for #82402

Since we already know which metadata ids are in significant domains when we have a list of entity_ids we can avoid the join of the states_meta table when generating statistics. Its also much faster since the database only has to do a hash lookup of some integers.

This will also improve the performance almost every multi-entity history query since nothing in production selects without a list of entities anymore (the old history page was the only thing that did)

Type of change

  • [ ] Dependency upgrade
  • [ ] Bugfix (non-breaking change which fixes an issue)
  • [ ] New integration (thank you!)
  • [ ] New feature (which adds functionality to an existing integration)
  • [ ] Deprecation (breaking change to happen in the future)
  • [ ] Breaking change (fix/feature causing existing functionality to break)
  • [x] Code quality improvements to existing code or addition of tests

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to issue:
  • Link to documentation pull request:

Checklist

  • [x] The code change is tested and works locally.
  • [ ] Local tests pass. Your PR cannot be merged unless tests pass
  • [ ] There is no commented out code in this PR.
  • [ ] I have followed the development checklist
  • [ ] I have followed the perfect PR recommendations
  • [ ] The code has been formatted using Black (black --fast homeassistant tests)
  • [ ] Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • [ ] The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • [ ] New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • [ ] For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
  • [ ] Untested files have been added to .coveragerc.

To help with the load of incoming pull requests:

bdraco avatar Mar 19 '23 11:03 bdraco

Hey there @home-assistant/core, mind taking a look at this pull request as it has been labeled with an integration (logbook) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of logbook can trigger bot actions by commenting:

  • @home-assistant close Closes the pull request.
  • @home-assistant rename Awesome new title Renames the pull request.
  • @home-assistant reopen Reopen the pull request.
  • @home-assistant unassign logbook Removes the current integration label and assignees on the pull request, add the integration domain after the command.

home-assistant[bot] avatar Mar 19 '23 11:03 home-assistant[bot]

Hey there @home-assistant/core, mind taking a look at this pull request as it has been labeled with an integration (recorder) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of recorder can trigger bot actions by commenting:

  • @home-assistant close Closes the pull request.
  • @home-assistant rename Awesome new title Renames the pull request.
  • @home-assistant reopen Reopen the pull request.
  • @home-assistant unassign recorder Removes the current integration label and assignees on the pull request, add the integration domain after the command.

home-assistant[bot] avatar Mar 19 '23 11:03 home-assistant[bot]

This will also improve the performance almost every multi-entity history query since nothing in production selects without a list of entities anymore (the old history page was the only thing that did)

Should we consider to remove that feature? History has thought us that any "select all" / "control all" etc always gets outgrown and is never needed.

balloob avatar Mar 20 '23 02:03 balloob

It's on my list of things to do consider later, but I didn't want to do it in the same release as a whole lot of database changes, because I was concerned about the risk

bdraco avatar Mar 20 '23 02:03 bdraco

This will also improve the performance almost every multi-entity history query since nothing in production selects without a list of entities anymore (the old history page was the only thing that did)

Should we consider to remove that feature? History has thought us that any "select all" / "control all" etc always gets outgrown and is never needed.

Will be done in https://github.com/home-assistant/core/pull/90992

bdraco avatar Apr 07 '23 03:04 bdraco