support append

[JosiahParry]

It appears that ArcGIS REST API supports an append and an addFeatures endpoint. Which should be used by default? At present add_features() uses the addFeatures endpoint. The append endpoint is not implemented.

What is the difference? When should one be used over another?

https://developers.arcgis.com/rest/services-reference/enterprise/add-features.htm https://developers.arcgis.com/rest/services-reference/enterprise/append-feature-service-.htm

[mmachir]

I believe there are a few distinctions between append and add:

• Typically I have seen people either use add_features/update_features/delete_features for making feature-level modifications (which can be applied across a large set of features but often have to be batched in those cases) OR use a truncate & append workflow, where data is wholesale deleted and replaced/updated (or this can be done without truncating). So I think that the methods can have slightly different use cases.
• Another distinction is the data format for adds. For add_features, I believe all features must be passed in JSON, whereas append can support passing in sqlite, shp, fgdb, csv, geojson, etc. data.

I would say that the methods are slightly different and advanced users tend to know which operation they need, so it may be worth implementing both ultimately. I would personally prioritize add_features() (and the corresponding delete and update) since you can work with those to approximate append, but can't use append to approximate the granularity of add_features, if that makes sense. Open to others thoughts on this.

For reference, the python api implements both: add_features, delete_features, and update_features are grouped into a single method, arcgis.FeatureLayer.edit_features(), and there is also arcgis.FeatureLayer.append(). The python api gives this warning about using add vs append: When making large number (250+ records at once) of edits, append should be used over edit_features to improve performance and ensure service stability.

[JosiahParry]

Okay, sounds good. I think we can probably close this and make two follow up issues: 1 to support append and the other to document (better than anywhere else lol) the distinction between add, delete, update, and append.

[JosiahParry]

@mmachir I want to move to support append though after we have an initial release. I think the standard addFeatures and updateFeatures will be sufficient. However I think this should be partitioned into two functions that I want your feedback on. I think we should have a distinction between upserting and appending from a function perspective.

• upsert_features()
• append_features()

Do you concur? The distinction here is that upsert will allow upsertion (inserting and updating) and append_features() will only add new features.

My preferance is to use only featureCollection json and use multithreading of the request via httr2::multi_req_perform(). I don't like the concept of writing a shapefile from R objects then uploading that.

[mmachir]

@JosiahParry just to clarify, do you mean that the upsert_features() and append_features() functions would use the addFeatures and updateFeatures REST API calls, rather than the append call? This could be my own ignorance and I may be thinking of the Python API here rather than the REST API, but I wonder if there would be issues with multithreading and creating ObjectID/unique IDs when writing features to the service concurrently? If this can be implemented in a way that allows for appending/upserting thousands (+) records smoothly, then I am all for it. Also totally agree with avoiding writing to shp- I believe append also supports geojson and some other formats that might be more workable. I do agree that having both append and upsert would be nice! Upsert is something that a lot of users ask for/are looking for, in my experience.

[JosiahParry]

Note that addFeatures and updateFeatures endpoints are supported. append is not, though.