great_expectations icon indicating copy to clipboard operation
great_expectations copied to clipboard

Issue connecting to GCS Json data

Open Iyanumanuel opened this issue 2 years ago • 2 comments

Describe the bug

My data source is stored on Google Cloud (GCS bucket) in JSON files and I used the guide here to create a datasource connection, but when I try to create an expectation suite by connecting to sample batch of json data I get a TypeError: BatchRequest.init() missing 1 required positional argument: 'data_asset_name' error

I tried to look online for guides on connecting GE to json datasource but all I found was CSV, any help on this? I have tried different json files and it still can't parse them.

I followed the steps in this guide but the guide used CSV batch data.

Please can anyone help with this?

Environment (please complete the following information):

  • Operating System: [Linux- Ubuntu 22.04)
  • Great Expectations Version: [ 0.15.21]

Iyanumanuel avatar Sep 17 '22 22:09 Iyanumanuel

Hi @Iyanumanuel - thanks for your question. I think the main issue here is that currentl Great Expectations only works on tabular data (and not JSON), and so your Data Connector is not picking up that there are any data files. Down the road, we hope to add support for unstructured data, but we don't have a firm timeline for that at this point in time.

talagluck avatar Sep 22 '22 18:09 talagluck

Oh great, thanks for the reply @talagluck.

I hope this can be done soon enough.

Iyanumanuel avatar Sep 23 '22 09:09 Iyanumanuel

Hi @Iyanumanuel - thanks again for raising this. We likely won't be able to prioritize this in the short-term, but I've logged this feedback. We're going to close this issue for now, but if anything new comes up, please feel free to add context and re-open.

talagluck avatar Mar 10 '23 14:03 talagluck