hapifhir
Problem loading ValueSet from Belgium's AllergyIntolerance IG
Hello, I'm currently investigating compatibility between the hapi-fhir-jpaserver-starter and Belgium's IGs but I'm currently having problems with this specific ValueSet: https://www.ehealth.fgov.be/standards/fhir/allergy/ValueSet-be-causativeagent.html
The ValueSet looks like this:
"resourceType": "ValueSet",
"id": "be-causativeagent",
"text": {
"status": "generated",
"div": "<div xmlns=\"http://www.w3.org/1999/xhtml\"><p>This value set includes codes based on the following rules:</p><ul><li>Include codes from <a href=\"http://www.snomed.org/\"><code>http://snomed.info/sct</code></a> where concept in 50851000172106 (Belgian subset for AllergyIntolerance, causative agent, non-drug)</li><li>Include codes from <a href=\"http://www.snomed.org/\"><code>http://snomed.info/sct</code></a> where concept in 50841000172109 (Belgian subset for AllergyIntolerance, causative agent, drug)</li></ul></div>"
},
"extension": [
{
"url": "http://hl7.org/fhir/StructureDefinition/structuredefinition-fmm",
"valueInteger": 1
}
],
"url": "https://www.ehealth.fgov.be/standards/fhir/allergy/ValueSet/be-causativeagent",
"version": "1.2.0",
"name": "BeCausativeAgent",
"title": "BeCausativeAgent",
"status": "active",
"experimental": false,
"date": "2023-12-18T12:02:54+01:00",
"publisher": "eHealth Platform",
"contact": [
{
"name": "eHealth Platform",
"telecom": [
{
"system": "url",
"value": "https://www.ehealth.fgov.be/standards/fhir"
},
{
"system": "email",
"value": "[email protected]"
}
]
},
{
"name": "Message Structure eHealth",
"telecom": [
{
"system": "email",
"value": "[email protected]",
"use": "work"
}
]
}
],
"description": "Causative Agents for Allergy (non medication)",
"jurisdiction": [
{
"coding": [
{
"system": "urn:iso:std:iso:3166",
"code": "BE",
"display": "Belgium"
}
]
}
],
"copyright": "This value set includes content from SNOMED CT, which is copyright © 2002+ International Health Terminology Standards Development Organisation (SNOMED International), and distributed by agreement between SNOMED International and HL7. Implementer use of SNOMED CT is not covered by this agreement",
"compose": {
"include": [
{
"system": "http://snomed.info/sct",
"version": "http://snomed.info/sct/11000172109",
"filter": [
{
"property": "concept",
"op": "in",
"value": "50851000172106"
}
]
},
{
"system": "http://snomed.info/sct",
"version": "http://snomed.info/sct/11000172109",
"filter": [
{
"property": "concept",
"op": "in",
"value": "50841000172109"
}
]
}
]
}
}
It should take the values from 2 subsets found in the latest SNOMED CT Belgian Edition (http://snomed.info/sct/11000172109):
- SCTID 5085100017210: Belgian subset for AllergyIntolerance, causative agent, non-drug (https://browser.ihtsdotools.org/?perspective=full&conceptId1=50851000172106&edition=MAIN/SNOMEDCT-BE/2024-12-15&release=&languages=en,nl,fr,de)
- SCTID 50841000172109: Belgian subset for AllergyIntolerance, causative agent, drug (https://browser.ihtsdotools.org/?perspective=full&conceptId1=50841000172109&edition=MAIN/SNOMEDCT-BE/2024-12-15&release=&languages=en,nl,fr,de)
I'm importing Beglgium's AllergyIntolerance IG like this in the application.yaml file:
(I'm also importing the Belgian Core IG, not in the image)
After a while I should expect all of my ValueSet to be expanded, but BeCausativeAgent says "FAILED TO EXPAND":
And this is the error I get:
2025-01-08T18:58:15.046+01:00 ERROR 41236 --- [ler-clustered-1] ca.uhn.fhir.jpa.term.TermReadSvcImpl : Failed to pre-expand ValueSet with URL[https://www.ehealth.fgov.be/standards/fhir/allergy/ValueSet/be-causativeagent]: HAPI-0888: org.hl7.fhir.common.hapi.validation.support.InMemoryTerminologyServerValidationSupport$ExpansionCouldNotBeCompletedInternallyException: HAPI-0702: Unable to expand ValueSet because CodeSystem could not be found: http://snomed.info/sct|http://snomed.info/sct/11000172109
ca.uhn.fhir.rest.server.exceptions.InternalErrorException: HAPI-0888: org.hl7.fhir.common.hapi.validation.support.InMemoryTerminologyServerValidationSupport$ExpansionCouldNotBeCompletedInternallyException: HAPI-0702: Unable to expand ValueSet because CodeSystem could not be found: http://snomed.info/sct|http://snomed.info/sct/11000172109
at ca.uhn.fhir.jpa.term.TermReadSvcImpl.expandValueSetHandleIncludeOrExclude(TermReadSvcImpl.java:1051)
at ca.uhn.fhir.jpa.term.TermReadSvcImpl.lambda$doExpandValueSet$2(TermReadSvcImpl.java:934)
at org.springframework.transaction.support.TransactionOperations.lambda$executeWithoutResult$0(TransactionOperations.java:68)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
at org.springframework.transaction.support.TransactionOperations.executeWithoutResult(TransactionOperations.java:67)
at ca.uhn.fhir.jpa.term.TermReadSvcImpl.doExpandValueSet(TermReadSvcImpl.java:934)
at ca.uhn.fhir.jpa.term.TermReadSvcImpl.expandValueSet(TermReadSvcImpl.java:896)
at ca.uhn.fhir.jpa.term.TermReadSvcImpl.preExpandDeferredValueSetsToTerminologyTables(TermReadSvcImpl.java:2461)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:355)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:716)
at ca.uhn.fhir.jpa.term.TermReadSvcImpl$$SpringCGLIB$$0.preExpandDeferredValueSetsToTerminologyTables(<generated>)
at ca.uhn.fhir.jpa.term.TermReadSvcImpl$Job.execute(TermReadSvcImpl.java:3226)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
Is there a correct way of importing this kind of ValueSet? Is HAPI compatible with this kind of ValueSet?
Thanks,
Marcelo
Do you have a CodeSystem resource with matching System and Versions from the ValueSet ?
"system": "http://snomed.info/sct",
"version": "http://snomed.info/sct/11000172109",
(That version looks weird to me. I'm not familiar with snomed versioning, so it may be correct, but it looks odd)
@XcrigX that use of SNOMED looks correct to me. @marceloabrumet - can you update and try again with version 8.2 and enable remote terminology with enabling this: https://github.com/hapifhir/hapi-fhir-jpaserver-starter/blob/0b907a4121f9737ba24a4d786b401814926cb819/src/main/resources/application.yaml#L302 ?
Hello Jens, I updated to 8.2.0 as you suggested and I uncommented the Remote Terminilogy config part like this:
remote_terminology_service: # all: # system: '*' # url: 'https://tx.fhir.org/r4/' snomed: system: 'http://snomed.info/sct' url: 'https://tx.fhir.org/r4/' # loinc: # system: 'http://loinc.org' # url: 'https://hapi.fhir.org/baseR4/'
I launched my server and when I execute GET /ValueSet/be-causativeagent/$expand I still receive this error:
{ "resourceType": "OperationOutcome", "issue": [ { "severity": "error", "code": "processing", "diagnostics": "HAPI-0888: org.hl7.fhir.common.hapi.validation.support.InMemoryTerminologyServerValidationSupport$ExpansionCouldNotBeCompletedInternallyException: HAPI-0702: Unable to expand ValueSet because CodeSystem could not be found: http://snomed.info/sct|http://snomed.info/sct/11000172109" } ] }
I have the impression that if the Terminology Server's Belgian Edition CodeSystem is not named exactly as my "version" value it won't work.
Anyways, just to test if the operator in the filter was working I removed the version field from my ValueSet and made the same request and got this:
{ "resourceType": "OperationOutcome", "issue": [ { "severity": "error", "code": "processing", "diagnostics": "HAPI-0894: Don't know how to handle op=IN on property concept" } ] }
I think I'll have to wait for 8.4.0 to correctly test my ValueSet
So in 8.2.0, RemoteTerminologyServiceValidationSupport does not support remote expansion of ValueSets ... if you fiddle a little around and add the proper features and extend the RemoteTerminologyServiceValidationSupport with the following
@Override
public ValueSetExpansionOutcome expandValueSet(ValidationSupportContext theValidationSupportContext, @org.jetbrains.annotations.Nullable ValueSetExpansionOptions theExpansionOptions, @NotNull IBaseResource theValueSetToExpand) {
IGenericClient client = provideClient();
FhirContext fhirContext = getFhirContext();
IBaseParameters params = ParametersUtil.newInstance(fhirContext);
if (theValueSetToExpand != null) {
ParametersUtil.addParameterToParameters(fhirContext, params, "valueSet", theValueSetToExpand);
}
ParametersUtil.addParameterToParametersBoolean(fhirContext, params, "excludeNotForUI", true);
Parameters outcome = (Parameters) client.operation()
.onType("ValueSet")
.named("$expand")
.withParameters(params)
.execute();
return new ValueSetExpansionOutcome(outcome.getParameterFirstRep().getResource());
}
you can end up getting the following results if you invoke the expansion locally, that your HAPI FHIR instance then forwards to a TX server of your choice (I picked https://tx.fhir.org/r4/):
{
"resourceType": "OperationOutcome",
"text": {
"status": "generated",
"div": "<div xmlns=\"http://www.w3.org/1999/xhtml\"><p>The value set 'https://www.ehealth.fgov.be/standards/fhir/allergy/ValueSet/be-causativeagent-add|1.2.0' expansion has 7503 codes, which is too many to display (>1000)</p></div>"
},
"issue": [
{
"severity": "error",
"code": "too-costly",
"details": {
"text": "The value set 'https://www.ehealth.fgov.be/standards/fhir/allergy/ValueSet/be-causativeagent-add|1.2.0' expansion has 7503 codes, which is too many to display (>1000)"
},
"diagnostics": "0 0 : start\r\n15 15: not in cache\r\n15 0: tx-op\r\n15 0: 0ms start working\r\n15 0: 0ms compose #1\r\n15 0: 0ms compose #1\r\n15 0: 0ms prepare filters\r\n15 0: 0ms iterate filters\r\n47 32: 32ms iterate filters done\r\n47 0: 32ms prepare filters\r\n47 0: 32ms iterate filters\r\n281 234: 266ms iterate filters done\r\n281 0: 266ms finish up\r\n"
}
]
}
@patrick-werner / @jamesagnew - what do you think? Does it make sense to add remote valueset expansion as well as a configuration option?