cognite-sdk-python
cognite-sdk-python copied to clipboard
Make pandas conversion distribute over iterables
Is your feature request related to a problem? Please describe.
I was a bit surprised that
pd.concat((asset.to_pandas().T for asset in assets)) != assets.to_pandas()
since I would expect that the conversion of a collection should equal the collection of converted elements. For instance, I would expect that the following property would hold for all pandas.Series
:
ser = pd.Series([1, 2, 4, 5])
ser.map(str) == pd.Series(map(str, ser))
The inconsistency is due to one of them including metadata, while the other one does not.
assets = client.assets.list(limit=99999)
assets[0:1].to_pandas() # Single row, no metadata
assets[0].to_pandas().T # Single row, with metadata
Describe the solution you'd like
- Be consistent.
- Perhaps add an
expand_metadata
argument to theto_pandas
method allowing users to control behavior.
CogniteResource.to_pandas() has an expand
parameter, which takes a list of keys. Given that the corresponding values are dicts, it will expand the key-value pairs of each dict into their own rows. expand
defaults to ["metadata"], but can be set to an empty list if you want to keep the metadata column.
Didn't see that!
I would still argue that it should distribute. By the function signature there's no indication (1) what the argument means precisely, (2) how it can be used, or (3) what the default is. The default could for instance be expand = ("metadata",)
to avoid mutable parameters, or expand=None
with an explanation in the docstring.
help(assets[0].to_pandas)
Help on method to_pandas in module cognite.client.data_classes._base:
to_pandas(expand:List[str]=None, ignore:List[str]=None) method of cognite.client.data_classes.assets.Asset instance
Convert the instance into a pandas DataFrame.
Returns:
pandas.DataFrame: The dataframe.
Describe the solution you'd like
- Perhaps add an expand_metadata argument to the to_pandas method allowing users to control behavior.
Coming in the next major version: #1453