The integration cannot find user defined plants from open.plantbook.io

[mihsu81]

Hi @Olen,

First of all, thank you for your awesome integrations and for the time you put in to make them even better in the recent weeks.

User defined plants on open.plantbook.io are not returned when trying to add a plant and the below error is returned in the logs of HA. Tested with v2.0.0-beta.3. This was not an issue in v1.

Thanks in advance for your help in getting this issue sorted.

Log Details (ERROR)
This error originated from a custom integration.
Logger: aiohttp.server
Source: custom_components/plant/config_flow.py:172
Integration: Plant monitor ([documentation](https://github.com/Olen/homeassistant-plant/), [issues](https://github.com/Olen/homeassistant-plant/issues))
First occurred: 09:01:56 (3 occurrences)
Last logged: 09:08:30

Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py", line 435, in _handle_request
    resp = await request_handler(request)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_app.py", line 504, in _handle
    resp = await handler(request)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_middlewares.py", line 117, in impl
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 60, in security_filter_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 100, in forwarded_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 28, in request_context_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 82, in ban_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 236, in auth_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 136, in handle
    result = await result
  File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 177, in post
    return await super().post(request, flow_id)
  File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 73, in wrapper
    result = await method(view, request, data, *args, **kwargs)
  File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 109, in post
    result = await self._flow_mgr.async_configure(flow_id, data)
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 277, in async_configure
    result = await self._async_handle_step(
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 359, in _async_handle_step
    result: FlowResult = await getattr(flow, method)(user_input)
  File "/config/custom_components/plant/config_flow.py", line 172, in async_step_select_species
    self.plant_info[ATTR_SPECIES] = user_input[ATTR_SPECIES]
KeyError: 'species'

[slaxor505]

@mihsu81 I think this is partly because of the 'Plant Search' API call at this stage does not return user plants but only public plants. ('Plant Details' call correctly returns userPlant) I'm currently working on this from API end and I will release update to handle it shortly. Though, I'd not expect it to error out. I expect that you just cannot add your userPlants using new integration UI.

@Olen You may take a look why the error is here. What are the steps to get a plant? Do you search with input string and then get details for a particular plant?

[Olen]

Looks like I did not properly validate the resuts from OPB. You are right. When OPB search returned 0 results, you were stille allowed to select a species, but the result was obviously an "None", which did not play well with the next step.

Fixed now (along with a few other reported issues. Beta 4 coming up.

[mihsu81]

@Olen Working fine now with v2.0.0-beta.4. @slaxor505 Indeed it was not throwing an error in the UI, just didn't show any available plants and didn't allow me to continue. The error was showing in HA logs. Thanks again for your help. 😊

[Olen]

I'm reopening this just to keep track.

The issue in the title is correct and still valid even if the traceback was fixed.

User defined plants are not included in the default search results from Openplantbook.

As a temporary fix, you can just add the plant with default values, and then go into the configuration of the plant and do a "Force refresh from Openplantbook" (requires beta 7). As long as the species is typed exactly as the pid of your plant, it will be updated with the correct data from Openplantbook.

There is work going on to include the user defined plants in the search results which should fix this.

[mihsu81]

@Olen @slaxor505 To be honest with you, v2.0.0-beta.4 solved the issue of the user defined plants. I have specifically checked if @slaxor505 had changed the flowers from user defined to generally available before updating and adding the plant. Or maybe this is about a subject I don't understand. 😁

[Olen]

@mihsu81 I'll try to explain in a bit more detail.

Openplantbook provides two methods of accessing data.

search - which is a general search, and will return a list of zero or more matches. This is what is used when you set up a new plant in the new integration, and this generates a drop down menu where you select the exact species. So when you set up a new plant, you can type e.g. "Red Rose" as the species, and it will create a dropdown of all species in Openplantbook matching that string.
But this search does not include user defined plants. So if you had created a "My Red Rose", it would not be on the list, and you can not select it from the dropdown.

• Now, the error you saw was not directly related to the missing user defined plants, but indirectly, because I was not correctly handling the case where the list from Openplantbook was empty. It generated an empty dorpdown, which then caused further errors. So if you searched for "My Red Rose", Openplantbook would return an empty list, and that caused the error. So I fixed handling of empty lists, but that itself does not help with the fact that user defined plants are not in the list in the first place.

The other method Openplantbook provides is a get. This method requires an exact match of the pid (I believe it is called Scientific name when you create you own plants in Openplantbook), and returns the details for that unique entry in the database. This method works with both public plants and user defined plants.
So a get with "my red rose" as input will return the details of your user defined plant even if the search did not list it. This method is used when getting the details of a plant when you set it up, and when you update a plant (but as mentioned, it requires an exact match to get the data). So if whatever you typed in the "Species" field when you set up a new plant was exactly what you have entered as the scientiific name in Openplantbook, you would not get a list of plants, but you would get direclty to the plant details setup, where your user defined details would be fetched from Openplantbook. So in that sense, the user defined plants works, but you just need to be very cautious when you type in the species in the first form.

The thing I/we want to fix is that the user defined plants should be included in the search results. That way, your "My Red Rose" would be an option in the dropdown also if you just search for "Red Rose". Maybe a minor detail, but from a usabilitiy view, I think it is important enough to keep the issue open for now.

I hope that clears things up even more.

And thank you for testing

[mihsu81]

@Olen Thank you for taking the time to explain it. Makes perfect sense.

[alexiri]

@mihsu81 I think this is partly because of the 'Plant Search' API call at this stage does not return user plants but only public plants. ('Plant Details' call correctly returns userPlant) I'm currently working on this from API end and I will release update to handle it shortly. Though, I'd not expect it to error out. I expect that you just cannot add your userPlants using new integration UI.

@slaxor505 is the API component open source? Perhaps we could help implement this feature.

[slaxor505]

Good timing, I'm releasing this feature today :)

See release notes: https://github.com/slaxor505/OpenPlantbook-client

[slaxor505]

It's been released. Please test if it works now. Thanks for offering help @alexiri

[Olen]

Since nobody has replied for almost a year, I guess it works now