fhir-works-on-aws-deployment
fhir-works-on-aws-deployment copied to clipboard
awslabs
[Misc] Interest in pre/post operation hooks
What's on your mind? We are working on adding decoupled pre/post hooks for all of the FHIR operations internal to the fhirServer. The problems we're trying to solve are the ability to wire up logic to run when FHIR operations are occurring but not have to pollute the endpoint logic with n number of mutually exclusive operations. There's a long tail of problems that can't be solved by Subscriptions that hooks would solve. Namely, you need access to read operations, you want to mutate the request or response in a systematic fashion or you want to make sure your logic runs before the HTTP response is sent.
Here are some examples we are going to wire up to pre/post FHIR operation hooks.
- Create a security stream of all operations to support compliance to regulations and security analysis tooling. Subscriptions only publishes the writes and not reads.
- Writing authz updates based on FHIR resource instance changes before the FHIR response is returned to the calling client.
- operation metrics
Gauging if there would be interest in this being part of the upstream solution. There's nothing proprietary about the implementation of these operation hooks and the code change is relatively straight forward since the endpoint logic is housed in the handlers. Happy to do the extra work here to open source or if it's not solving a problem for others, we can keep in our internal fork as well.
Versions (please complete the following information):
- Release Version installed v2.5.1-smart
Hi @Zambonilli,
Thanks for reaching out! I'll bring this idea to my team and see if we have customer interests here. At the meantime, it would be really helpful if there's any other documentation(e.g. design doc, architecture diagram) you could share with us.
I like the idea of hooks here as it would provide easy extension points for different use cases. Just so I understand it correctly - these hooks are designed for backend functionalities only(metrics, analytics, compliance), they should not interfere with how the FHIR server behaves for end customer and are not related to any FHIR specifications?
Thanks, Yanyu
Yeah, it would be similar behavior as pre/post hooks or interceptors in an ORM to inject logic around database operations. People can register their async function to run right before or right after the persistence or search call. We struggled with the control flow questions. We looked at implementing a distributed transaction logic but the FHIR bundle batch operation would have been a beast to manage. We also looked at the opposite spectrum, where these hooks are just a node eventemitter. The EventEmitter solution has the problem that a subscribe cannot block the processing and the reliability is only as good as the process exiting.
Ultimately, we ended up creating a publish subscribe interface that allows for a subscriber to have their logic cause the calling code wait for completion and the response includes a success boolean that can change control flow to throw and result in a 500. The thought was that it'd be better to let a subscriber cause failure so use cases like an authz write not succeeding results in a client not knowing something failed. The downside is that there was a partial failure in processing with the FHIR operation succeeding but the subscriber failing and the client might not be able to recover the failure, in this case retry if it's a create.
We don't have any design diagrams but I can share some screenshots of the interface, service code and a basic subscriber that should give a good idea of the mechanics.
Hey @Zambonilli Would you be open to getting on a call to discussing this with us? Anyone else you want to include from your side?
I've been able to get feature branches with support for pre/post FHIR operation hooks ported from our private fork. Turns out it's a lot harder to go upstream from a private fork than pull upstream down, especially when you're a version behind. I was able to get the fhirServer running locally using yarn link, booting and successfully performing operations with pre/post hooks firing successfully. I've included the pre/post hook subscribers that are not specific to us. They are an example console log subscriber and a subscriber that sends operation counts to cloudwatch metrics under the FWoA namespace. The latter is enabled if you set the OP_METRIC_SUBSCRIBER_ENABLED environment variable to true.
I didn't get enough time to truly debug and regression test the port itself but we have had this feature on in our non-prod environments for a couple of weeks. I'm out on holiday tomorrow but will pick this back up on Monday. However, I wanted to get the code available for the FWoA team for review and the ability to run locally sooner rather than later so we can get some feedback. Again, happy if this needs work or doesn't make sense to pull upstream. Just let us know.
interface = https://github.com/awslabs/fhir-works-on-aws-interface/compare/mainline...ResMedOSS:fhir-works-on-aws-interface:feature/Issue_646
routing = https://github.com/awslabs/fhir-works-on-aws-routing/compare/mainline...ResMedOSS:fhir-works-on-aws-routing:feature/issue_646
deployment = https://github.com/awslabs/fhir-works-on-aws-deployment/compare/mainline...ResMedOSS:fhir-works-on-aws-deployment:feature/issue_646
FHIR Works on AWS has been moved to maintenance mode. While in maintenance, we will not add any new features to this solution. All security issues should be reported directly to AWS Security at [[email protected]] (mailto:[email protected]). If you are new to this solution, we advise you to explore using [HealthLake] (https://aws.amazon.com/healthlake), which is our managed service for building FHIR based transactional and analytics applications. You can get started by contacting your AWS Account team. If you are an existing customer of FHIR Works on AWS, and have additional questions or need immediate help, please reach out to [email protected] or contact your AWS Account team.
