to_featureset() is not handling fields with null date values correctly

[tbrobin]

trafficstars

I am using to_featureset() to create a featureset from a spatially enabled dataframe and then using edit_features() to edit a feature layer on AGOL. I was having this issue https://github.com/Esri/arcgis-python-api/issues/1693 so I rolled back to a previous version of the API. Now that 2.3.0 is out, I was going to try my luck again.

It seems there is a new bug with to_featureset(). Only some of my features manage to successfully make it to the feature layer using edit_features(). And I am getting this message when they fail.

{'addResults': [{'objectId': 1, 'uniqueId': 1, 'globalId': None, 'success': False, 'error': {'code': 1000, 'description': 'The incoming tabular data stream (TDS) remote procedure call (RPC) protocol stream is incorrect. Parameter 21 ("@applicant_closeout_date"): The supplied value is not a valid instance of data type float. Check the source data for invalid values. An example of an invalid value is data of numeric type with scale greater than precision.'}}], 'updateResults': [], 'deleteResults': []}

That field is not a float. It is a date with null values mixed in. I did some troubleshooting and it seems to_featureset() is treating that field differently than other date fields in the data with no nulls.

Here is how it treats that field if I remove the null values, and these values will work in a bulk edit using edit_features(adds=featureset.features).

Here is how it treats that field if it has null values, and it won't work using edit_features(adds=featureset.features).

[nanaeaubry]

@tbrobin Thanks for reporting this, sorry for the delayed response. I will take a look and see where this new error is coming from. Ill post any updates!

[nanaeaubry]

@tbrobin Can you provide us with some reproducible data and code?

[tbrobin]

Dates.xlsx `` import pandas as pd from arcgis.features import GeoAccessor, GeoSeriesAccessor

#path to Dates excel dates_excel=

nonulls=pd.read_excel(dates_excel,sheet_name='NoNulls') withnulls=pd.read_excel(dates_excel,sheet_name='WithNulls')

nonulls_fs=nonulls.spatial.to_featureset() withnulls_fs=withnulls.spatial.to_featureset()

#test feature set with nulls print(withnulls_fs.features)

#test feature set without nulls print(nonulls_fs.features) `

[tbrobin]

Thanks for looking into this!

[nanaeaubry]

@tbrobin Ok I see the issue, Pandas interprets NaN values as float and not int. Hence why the entire column gets changed to float values when null dates are present. A workaround for now would be to take all the values resulting from the feature set and type cast them as int. We are adding a fix in the code for this, thanks for reporting it!

[tbrobin]

Great! So will this be fixed in the next ArcGIS API Release?

[nanaeaubry]

@tbrobin yes!

[tbrobin]

Hi, I noticed the June AGOL Update came out, but there was no ArcGIS Python API update. When is the next API update that will have this fix?

[nanaeaubry]

@tbrobin We do not follow the same schedule as ArcGIS Online so our next update won't be until the Fall. Our releases are more in line with Enterprise and Pro releases.

[tbrobin]

I saw a new version of the API was released on 7/9, but there was no fix. The null date values are still causing the issue with to_featureset(). I still have to be on API version 2.1 for my script to work properly.

[nanaeaubry]

@tbrobin Yes, this release was for arcgis_learn only unfortunately. The fix will be in the Fall release

[tbrobin]

Got it. Thank you!