Add integration tests against pystac-client

[geospatial-jeff]

This API should work well with pystac-client. We need integration tests to make sure this is the case.

https://github.com/stac-utils/pystac-client

First brought up in #207.

[geospatial-jeff]

Ideally these tests are done against the API layer instead of individual backends.

[martys]

1 up - as currently the API is successfully spitting out responses that pystac library is not able to parse. In other words the responses are not STAC compliant :(
(in our case license field was missing)

[gadomski]

In other words the responses are not STAC compliant :( (in our case license field was missing)

@martys, can you provide an reproducable example? There's validation in the test suite so it's surprising to me that stac-fastapi is not producing valid STAC.

Note that if you're using the fields extension, you might not be returning valid STAC by design.

[martys]

@gadomski - sure. Here's an example of a collection that we have in our database, that is successfully returned by the API - and that the PyStac client refuses to parse, due to the lack of 'license' field

{
    "id": "elsegundo_powerpole_drone_deploy_rgb",
    "type": "Collection",
    "title": "ElSegundo Powerpoles - DroneDeploy (RGB)",
    "extent": {
      "spatial": {
        "bbox": [
          [
            -118.423123,
            33.906993,
            -118.401384,
            33.916271
          ]
        ]
      },
      "temporal": {
        "interval": [
          [
            "2021-01-01T21:39:14Z",
            "2021-06-30T08:15:54Z"
          ]
        ]
      }
    },
    "keywords": [
      "DroneDeploy",
      "United States",
      "ElSegundo",
      "Powerpoles"
    ],
    "platform": "Unmanned Drones",
    "serviceId": "1500001156",
    "short_desc": "Drone images of powerpoles of ElSegundo, captured via dronedeploy",
    "description": "Drone images of powerpoles of ElSegundo, captured via dronedeploy",
    "stac_version": "1.0.0",
    "links": [
      {
        "rel": "items",
        "type": "application/geo+json",
        "href": "https://api.azure.com/collections/elsegundo_powerpole_drone_deploy_rgb/items"
      },
      {
        "rel": "parent",
        "type": "application/json",
        "href": "https://api.azure.com/"
      },
      {
        "rel": "root",
        "type": "application/json",
        "href": "https://api.azure.com/"
      },
      {
        "rel": "self",
        "type": "application/json",
        "href": "https://api.azure.com/collections/elsegundo_powerpole_drone_deploy_rgb"
      }
    ]
  }

[gadomski]

The license field is required on Collections ... did you have a license attribute on the collection when you added it to the database?

stac-fastapi does not validate STAC items before returning them as a part of request. This is a conscious decision, since validation can be slow operation.

[martys]

hm. Interestingly we're using the pystac lib to create catalog and collections in the first place - assuming the use of dedicated objects would ensure that INVALID objects would not get created and stored into the DB. but.. apparently the following was not sufficient to prevent invalid data from getting into the database

 try:
        collection = pystac.Collection(
            id=id,
            title=title,
            description=description,
            extent=extent,
            keywords=keywords,
            extra_fields=dict(
                serviceId=serviceId,
                platform=platform,
                short_desc=short_desc,
            )
        )
        
    except Exception as ex:
        logger.exception(f"exception occurred: {ex}")
        raise PystacUtilsError(f"unable to create collection ({id})")

[gadomski]

Hm, that's surprising to me as well, since license defaults to proprietary: https://pystac.readthedocs.io/en/stable/api/pystac.html?highlight=collection#pystac.Collection. What version of pystac are you using?

You can always run collection.validate() before storing to ensure it's valid (assuming you've installed pystac with jsonschema support, e.g. pip install "pystac[validation]").

[jonhealy1]

Closing but feel free to re-open.