fhir-data-pipes icon indicating copy to clipboard operation
fhir-data-pipes copied to clipboard

Published 20 hours ago • verified publisher icon google

Update documentation to clarify feature roadmap (e.g. what is ready, what is not yet ready).

Open jjtswan opened this issue 1 year ago • 3 comments

jjtswan avatar Feb 28 '23 22:02 jjtswan

For example, I think we want to indicate that we want to enable support of single-machine deployment using FHIR search, instead of the native HAPI integration - but that it is not yet ready.

jjtswan avatar Feb 28 '23 22:02 jjtswan

My understanding of current features:

Pipeline

Transform data from a FHIR-based data source to Parquet files or a different FHIR store.

  • Batch Mode
    • FHIR Search-based source
      • Generic FHIR, HAPI, OpenMRS
    • JDBC-based source
      • HAPI, OpenMRS
  • Streaming Mode
    • Source: OpenMRS
  • Output options
    • Parquet files
    • FHIR API sink
      • Generic FHIR
    • JDBC sink [deprioritized?]
      • HAPI only

Controller

Schedule incremental runs for a Pipeline to Parquet files. Provides a GUI to run Pipeline, see its status, and see its settings.

  • Input - JDBC-based HAPI FHIR server
  • Output - Parquet files

Single Machine Deployment

A single Docker Compose configuration to run Controller and query the Parquet files via a Spark thrift server.

  • Input - JDBC-based HAPI FHIR server
  • Output - Spark thrift server at port jdbc:hive2://server:10001

Test docker images

Docker images to try out Pipelines with different sources and sinks.

  • Input
    • HAPI FHIR server configured with a Postgres server; no preloaded data
    • OpenMRS server configured with MySQL
  • Sink
    • HAPI FHIR server
  • DWH
    • OpenHIM
    • Indicator Calc

Synthea HIV

Generate synthetic HIV patient data using Synthea. Upload data to HAPI, GCP, or OpenMRS. Use pre-generated test data in your own development.

  • Generator
    • Input: # patients
    • Output: 1 FHIR Bundle per generated patient history
  • Uploader
    • HAPI
    • GCP
    • OpenMRS
  • Sample Data
    • 79 patients, 4006 Encounters, and 17279 Observations @ ~100MB

DWH Query Library [deprioritized?]

Simplifies querying of FHIR-based data warehouses by providing a unified query API across Spark and BigQuery.

williamito avatar Mar 08 '23 21:03 williamito

This is not a requirement for Beta launch so moving to "post-beta" milestone.

bashir2 avatar May 30 '23 15:05 bashir2