Unable to filter dates in Jupyter (hass pyscript kernel)

[yozik04]

Hello. Thank you for a great integration!

As electricity price got up I am working on a script that would control my heating depending on current electricity price.

I have faced the problem that I am unable to filter out today's prices from the list.

I have double checked next test case in regular Python 3 (ipykernel) where it works.

Test case to reproduce filtering bug in hass pyscript kernel

import datetime
import zoneinfo

data = [{'start': datetime.datetime(2021, 10, 24, 0, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 1, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.114}, {'start': datetime.datetime(2021, 10, 24, 1, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 2, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.068}, {'start': datetime.datetime(2021, 10, 24, 2, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 3, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.081}, {'start': datetime.datetime(2021, 10, 24, 3, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 4, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.079}, {'start': datetime.datetime(2021, 10, 24, 4, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 5, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.079}, {'start': datetime.datetime(2021, 10, 24, 5, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 6, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.079}, {'start': datetime.datetime(2021, 10, 24, 6, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 7, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.085}, {'start': datetime.datetime(2021, 10, 24, 7, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 8, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.084}, {'start': datetime.datetime(2021, 10, 24, 8, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 9, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.085}, {'start': datetime.datetime(2021, 10, 24, 9, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 10, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.089}, {'start': datetime.datetime(2021, 10, 24, 10, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 11, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.1}, {'start': datetime.datetime(2021, 10, 24, 11, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 12, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.099}, {'start': datetime.datetime(2021, 10, 24, 12, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 13, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.105}, {'start': datetime.datetime(2021, 10, 24, 13, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 14, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.088}, {'start': datetime.datetime(2021, 10, 24, 14, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 15, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.074}, {'start': datetime.datetime(2021, 10, 24, 15, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 16, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.072}, {'start': datetime.datetime(2021, 10, 24, 16, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 17, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.092}, {'start': datetime.datetime(2021, 10, 24, 17, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 18, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.111}, {'start': datetime.datetime(2021, 10, 24, 18, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 19, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.125}, {'start': datetime.datetime(2021, 10, 24, 19, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 20, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.179}, {'start': datetime.datetime(2021, 10, 24, 20, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 21, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.163}, {'start': datetime.datetime(2021, 10, 24, 21, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 22, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.116}, {'start': datetime.datetime(2021, 10, 24, 22, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 24, 23, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.09}, {'start': datetime.datetime(2021, 10, 24, 23, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 0, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.084}, {'start': datetime.datetime(2021, 10, 25, 0, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 1, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.072}, {'start': datetime.datetime(2021, 10, 25, 1, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 2, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.086}, {'start': datetime.datetime(2021, 10, 25, 2, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 3, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.088}, {'start': datetime.datetime(2021, 10, 25, 3, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 4, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.049}, {'start': datetime.datetime(2021, 10, 25, 4, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 5, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.051}, {'start': datetime.datetime(2021, 10, 25, 5, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 6, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.088}, {'start': datetime.datetime(2021, 10, 25, 6, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 7, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.096}, {'start': datetime.datetime(2021, 10, 25, 7, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 8, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.117}, {'start': datetime.datetime(2021, 10, 25, 8, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 9, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.156}, {'start': datetime.datetime(2021, 10, 25, 9, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 10, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.156}, {'start': datetime.datetime(2021, 10, 25, 10, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 11, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.142}, {'start': datetime.datetime(2021, 10, 25, 11, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 12, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.134}, {'start': datetime.datetime(2021, 10, 25, 12, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 13, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.133}, {'start': datetime.datetime(2021, 10, 25, 13, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 14, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.133}, {'start': datetime.datetime(2021, 10, 25, 14, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 15, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.126}, {'start': datetime.datetime(2021, 10, 25, 15, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 16, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.133}, {'start': datetime.datetime(2021, 10, 25, 16, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 17, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.138}, {'start': datetime.datetime(2021, 10, 25, 17, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 18, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.154}, {'start': datetime.datetime(2021, 10, 25, 18, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 19, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.18}, {'start': datetime.datetime(2021, 10, 25, 19, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 20, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.247}, {'start': datetime.datetime(2021, 10, 25, 20, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 21, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.18}, {'start': datetime.datetime(2021, 10, 25, 21, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 22, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.156}, {'start': datetime.datetime(2021, 10, 25, 22, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 25, 23, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.122}, {'start': datetime.datetime(2021, 10, 25, 23, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'end': datetime.datetime(2021, 10, 26, 0, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Tallinn')), 'value': 0.116}]

def is_today(x):
    today = datetime.date.today()
    return today == x["start"].date()

def filter_today(candidates):
    return filter(is_today, candidates)

result = list(filter_today(data))
expected = 24

assert expected == len(result), f"Should be {expected}. Got {len(result)}"

result

I am getting 48. But in real python it produces 24 results.

[dlashua]

It it too difficult to use your particular test case (imports, date changes, etc). However, I'm pretty sure you need native python functions for filter to work as expected. try adding the @pyscript_compile decorator on is_today and filter_today.

The following code (similar to yours but simplified) worked for me:

data = range(0,100)

@pyscript_compile
def is_zero(x):
    return 0 == x

@pyscript_compile
def filter_zero(candidates):
    return filter(is_zero, candidates)

result = list(filter_zero(data))
expected = 1

assert expected == len(result), f"Should be {expected}. Got {len(result)}"

log.info(result)

[ALERTua]

@yozik04 could you please update or close the issue? Thank you