FHIR pipeline: a smart ETL to standardize health data

Fhir-Pipe is an ETL tool backed by IA solutions, which makes standardisation of health data easy. It extracts and converts data from SQL databases into the standardized health format FHIR, using mapping rules build from the pyrog interface.

Mapping rules

The mapping rules are provided through the pyrog graphql API. To each FHIR Resource (e.g. Patient) corresponds a set of rules, and each attribute of the ressource (like patient.name.firstname) has a mapping instruction which details which DATABASE/TABLE/COLUMN to select and which processing scripts to apply, as the data might need to be cleaned. The list of generic and health care specific scripts is available at arkhn/scripts.

Goal of fhir-pipe

Fhir-Pipe is an ETL which is able to parse complex mapping rules which may describe intricated relations between tables. It is agnostic of the type of SQL databases used as input for its extraction, and uses Spark to perform highly scalable processing of data. The FHIR objects created are stored in a distributed file system to meet strict performance requirements.

Get started

Read our guide to start standardizing health data!

Contribute

We have reported several issues with the label Good first issue which can be a good way to start! You can also join our Slack to contact us if you have trouble or questions :)

If you're enthusiastic about our project, :star: it to show your support! :heart:

License

Apache License 2.0