Documentation Snippet for Accessing Item-based Datastores

[FeralCatColonist]

trafficstars

There does not appear to be any code samples dealing with accessing and administering the bulk-publish functionality of an item-based Data Store within an Enterprise Portal. A small notebook dealing with this use-case would be great showing administrators how to keep their item-derived datastores synced instead of relying on the GUI button.

The function in question that seems like it would solve this issue is

• https://developers.arcgis.com/python/api-reference/arcgis.gis.toc.html#arcgis.gis._impl._datastores.PortalDataStore.publish_layers

However, the documentation for this is a bit confusing

• publish_layers is mentioned within the documentation 5 times, under PortalDataStore
  • once is layers() and four times in publish_layers()
  • https://developers.arcgis.com/python/api-reference/arcgis.gis.toc.html?highlight=publish_layers#arcgis.gis._impl._datastores.PortalDataStore
• issue #1135 shows that this class is not reachable, and users are directed to utilize gis.datastore instead
• the publish_layers() requires a srv_config
  • this looks like it could be the output of the gis.datastore.layers(); but this seems to be created at point of sale and I'm not even sure it is the same thing.
  • Perhaps the description could also link to a sample template or the possible parameters of a configuration?
  • Would this configuration dict also be where you'd set up things like editing capability and syncing? I can iterate through services easy enough, but it seems a step-removed from how it should operate.
• the publish_layers() requires a server_id
  • within the item-based construct, it seems that the sync happens in_place and this server_id shouldn't be necessary?
  • Trying to find the server_id, which is described by the text "server_name", is a bit difficult, the documentation shows it as a parameter for a few functions but not as an output from any. Is this the name of the ArcGIS Server box?
  • I tried to interrogate the items using gis.datastore.describe() but the server_id that is shown as optional within the documentation is required according to the error message

item_id = "fb9566c40cc544d09ed9d1170fa06b9c"
item_datastore = arcgis.gis.Item(gis_Portal, item_id)
gis_Portal.datastore.publish_layers(item_datastore, srv_config, server_id)

Describe the solution you'd like A sample notebook to walk through the steps of a sync process would be just fantastic as a minimum solution to this problem. Ideally, I'd like the documentation revised to include PortalDataStore into Datastore since that's how the functionality is presenting within the code. Generally the usage examples are cryptically minimalist within the datastore section as well.

Additional context The goal of these changes is to utilize the new bulk-publishing option with ArcGIS Portal in an automated way, including enabling sync, so that these services and be shared to our AGOL via portal-to-portal collaboration.

[achapkowski]

Assuming you know how to register a datastore Item, you can do this:

item = gis.content.get(<item id>)
server_list = gis.admin.federation.servers
server_id = server_list["servers"][0]["id"]
cfg = {'serviceName': '7a932', 'type': 'MapServer', 'capabilities': 'Map', 'extensions': [{'typeName': 'FeatureServer', 'capabilities': 'Query', 'enabled': 'true', 'properties': {'maxRecordCount': '3500'}}]}
gis.datastore.publish_layers(
                item=item, srv_config=cfg, server_id=server_id
            )

[FeralCatColonist]

That's super great! Thank you so much for responding quickly. This gets me much closer to where I want to be.

I edited a few of the capabilities and I received the following error, which seems to be based on some of the optional parameters that were not filled out (full text at bottom of message):

Exception: Publish layes failed.
Invalid value for serverFolder. Layers have been published from this data store before and the value is read-only. Passing null is valid.  
(Error Code: 400)

Back to the documentation thing. Again, I'm not sure that I could have gotten there with the documentation as it stands currently. I think that there is a lot of knowledge of the inner-workings of the ArcGIS REST and Enterprise infrastructure that are needed to make some of these cognitive leaps. Particularly for the bulk-publishing, I think that it would be useful to have an example that shows the python equivalent of the vanilla call that Enterprise is making when you flip the sync layers button.

• Line 2
  • The servers property doesn't reference the server_id nomenclature that is used throughout the rest of the documentation. For people that do not have a holistic understanding of the entirety of the API (the ctrl+f types, such as myself) would struggle to make that connection).
  • I think that a URL reference to this would be a great addition for those other functions that require the server_id, along with adding that it returns a dict.
• Line 3
  • This is straightforward, I was wondering why the first position was chosen and viewed the list by itself to see that structure and that it could populate additional servers based on a multimachine deployment
• Line 4
  • service configuration, I was still confused as to what the full template would look like
  • I searched for "arcgis api service configuration json" and came across the REST documentation, I think that would be helpful as either a direct URL reference or the starting point for a reference that lives within the ArcGIS API for Python documentation
  • https://developers.arcgis.com/rest/enterprise-administration/server/createservice.htm
  • I also took a look at the JSON behind individual REST services to get a better idea of what my ideal target (sync capabilities) should look like; I feel like I'm able to do all of this now based on your code snippet. ,Back to the documentation, there's a lack of instructions leaving some of these really useful features outside the reach of Python developers coming from the ArcPy pipeline.

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_26584/3451043616.py in <module>
      7 server_id = server_list["servers"][0]["id"]
      8 cfg = {'serviceName': 'Viewer', 'type': 'MapServer', 'capabilities': 'Map', 'extensions': [{'typeName': 'FeatureServer', 'capabilities': 'Query,Create,Update,Delete,Uploads,Editing,Sync,Extract', 'enabled': 'true', 'properties': {'maxRecordCount': '3500'}}]}
----> 9 gis_Portal.datastore.publish_layers(item=item, srv_config=cfg, server_id=server_id)

~\.conda\envs\esri\lib\site-packages\arcgis\gis\_impl\_datastores\_ds.py in publish_layers(self, item, srv_config, server_id, folder, server_folder, future)
    496             "serverFolder": server_folder,
    497         }
--> 498         res = self._con.post(url, params)
    499         if res["success"] == True:
    500             status = item.status()

~\.conda\envs\esri\lib\site-packages\arcgis\gis\_impl\_con\_connection.py in post(self, path, params, files, **kwargs)
   1077             file_name=file_name,
   1078             try_json=try_json,
-> 1079             force_bytes=kwargs.pop("force_bytes", False),
   1080         )
   1081 

~\.conda\envs\esri\lib\site-packages\arcgis\gis\_impl\_con\_connection.py in _handle_response(self, resp, file_name, out_path, try_json, force_bytes, ignore_error_key)
    623                     return data
    624                 errorcode = data["error"]["code"] if "code" in data["error"] else 0
--> 625                 self._handle_json_error(data["error"], errorcode)
    626             return data
    627         else:

~\.conda\envs\esri\lib\site-packages\arcgis\gis\_impl\_con\_connection.py in _handle_json_error(self, error, errorcode)
    646 
    647         errormessage = errormessage + "\n(Error Code: " + str(errorcode) + ")"
--> 648         raise Exception(errormessage)
    649 
    650     def post_multipart(self, path, params=None, files=None, **kwargs):

Exception: Publish layes failed.
Invalid value for serverFolder. Layers have been published from this data store before and the value is read-only. Passing null is valid.  
(Error Code: 400)

[FeralCatColonist]

Is this a bug? I'm passing server_folder=None and it says that it is an invalid value.

[FeralCatColonist]

What was the resolution on this?