Healthcare API for FHIR bundle processing is very slow.

[Ashish-Balhara]

I'm trying to use Google healthcare API (v1 beta) for FHIR Bundle processing and used the exact python code from this doc: healthcare-fhir-execute-bundle. Also I'm generating bundle examples from Synthetic Patient Population Simulator.
And When I executed the code, I found it takes approx. 20 minutes for just 45 patient bundles. Its not sustainable for our use case. we supposed to use this API for more than 1000 such records at least. I just want to know: Is it normal? What should I do to improve the performance?

Thanks!

[msampathkumar]

Hi @Ashish-Balhara Thanks for reaching out. I am checking with the mentioned FHIR code sample developers, for assistance on this. They might share accurate details on this.

In case, if you have any more details that could help us then please share it there. Thanks _ For this repository code reviewers(m-strzelczyk), you may check details @ http://b/247558027

[noerog]

Hi @Ashish-Balhara:

Synthea can create very large bundles, but the time you quoted seems excessive. Can you try the suggestions in our network latency best practices doc (https://cloud.google.com/healthcare-api/docs/how-tos/best-practices) and let us know if you find that the performance improves? The sample you referred to only executes a single bundle file at a time; have you tried executing bundles in parallel?

If you have a support contract with GCP then the best thing to do would be to file a bug through your support contact. If not, you can file a bug using the public issue tracker (https://cloud.google.com/support/docs/issue-trackers).

[nikklassen]

It's also worth noting that when Synthea generates FHIR transaction bundles it refers to common resources using "query references", i.e. Practitioner?identifier=http://hl7.org/fhir/sid/us-npi|999999647 (docs). Querying for these resources for every entry can be slow, especially with bundles of hundreds of resources. One option would be to use the bulk data settings on Synthea (ndjson files) instead of transaction bundles, and import the resources.

[Ashish-Balhara]

Thanks @noerog, @nikklassen for your suggestions, @nikklassen I am trying to use what you suggest me. I've used the bulk data settings and downloaded ndjson files, but I didn't find any way in HealthCare api documentation (https://cloud.google.com/healthcare-api/docs/samples/healthcare-create-resource) to create resource from ndjson file. if you have any idea, please do tell.

[nikklassen]

To create a resource from an ndjson file you'll either need to split it yourself on newlines, then use create for each line, or use the import API https://cloud.google.com/healthcare-api/docs/how-tos/fhir-import-export#importing_fhir_resources. Importing will be a lot faster/easier, but you'll miss out on some features like pubsub notifications. But for just getting the data into your FHIR store import is the way to go.

[dandhlee]

Thank you nikklassen@ and noero@ for helping take a look!

Seems like all the questions have been answered, I'll close the issue but if you run into further problems feel free to re-open this issue or open another issue.