SDMX icon indicating copy to clipboard operation
SDMX copied to clipboard

Use If-Modified-Since header for retrieving content constraint

Open wiibaa opened this issue 2 months ago • 1 comments

When using SDMX JAVA GUI on a dataset with heavy content constraint, the application is slowed down trying to retrieve several time the same constraint when moving between dimensions

Image

It seems that some provider, at least ESTAT, are respecting the If-Modified-Since HTTP header that could be used to effficiently cache the content locally. Do you think it is something that could benefit the application in general ?

$ curl -v -H "If-Modified-Since: Tue, 9 Dec 2025 07:28:00 GMT" https://ec.europa.eu/eurostat/api/comext/dissemination/
sdmx/2.1/contentconstraint/ESTAT/DS-045409
> GET /eurostat/api/comext/dissemination/sdmx/2.1/contentconstraint/ESTAT/DS-045409 HTTP/1.1
> Host: ec.europa.eu
> User-Agent: curl/8.17.0
> Accept: */*
> If-Modified-Since: Tue, 9 Dec 2025 07:28:00 GMT
>
* Request completely sent off
< HTTP/1.1 304 Not Modified
< Date: Tue, 09 Dec 2025 10:39:29 GMT
< Last-Modified: Tue, 18 Mar 2025 12:00:00 CET

Application logs after a few change of dimensions selection

Dec 09, 2025 11:35:56 AM it.bancaditalia.oss.sdmx.util.QueryRunner runQuery
INFO: Contacting web service with query: https://ec.europa.eu/eurostat/api/comext/dissemination/sdmx/2.1/contentconstraint/ESTAT/DS-045409
Dec 09, 2025 11:36:07 AM it.bancaditalia.oss.sdmx.util.QueryRunner runQuery
INFO: Contacting web service with query: https://ec.europa.eu/eurostat/api/comext/dissemination/sdmx/2.1/contentconstraint/ESTAT/DS-045409
Dec 09, 2025 11:36:17 AM it.bancaditalia.oss.sdmx.util.QueryRunner runQuery
INFO: Contacting web service with query: https://ec.europa.eu/eurostat/api/comext/dissemination/sdmx/2.1/contentconstraint/ESTAT/DS-045409
Dec 09, 2025 11:36:36 AM it.bancaditalia.oss.sdmx.util.QueryRunner runQuery
INFO: Contacting web service with query: https://ec.europa.eu/eurostat/api/comext/dissemination/sdmx/2.1/contentconstraint/ESTAT/DS-045409
Dec 09, 2025 11:37:08 AM it.bancaditalia.oss.sdmx.util.QueryRunner runQuery
INFO: Contacting web service with query: https://ec.europa.eu/eurostat/api/comext/dissemination/sdmx/2.1/contentconstraint/ESTAT/DS-045409
Dec 09, 2025 11:37:23 AM it.bancaditalia.oss.sdmx.util.QueryRunner runQuery
INFO: Contacting web service with query: https://ec.europa.eu/eurostat/api/comext/dissemination/sdmx/2.1/contentconstraint/ESTAT/DS-045409
Dec 09, 2025 11:37:37 AM it.bancaditalia.oss.sdmx.util.QueryRunner runQuery
INFO: Contacting web service with query: https://ec.europa.eu/eurostat/api/comext/dissemination/sdmx/2.1/contentconstraint/ESTAT/DS-045409
Dec 09, 2025 11:37:49 AM it.bancaditalia.oss.sdmx.util.QueryRunner runQuery
INFO: Contacting web service with query: https://ec.europa.eu/eurostat/api/comext/dissemination/sdmx/2.1/contentconstraint/ESTAT/DS-045409
Dec 09, 2025 11:39:57 AM it.bancaditalia.oss.sdmx.util.QueryRunner runQuery
INFO: Contacting web service with query: https://ec.europa.eu/eurostat/api/comext/dissemination/sdmx/2.1/contentconstraint/ESTAT/DS-045409

wiibaa avatar Dec 09 '25 10:12 wiibaa

Hi @wiibaa , thanks for spotting this. This behaviour is wrong for all providers that do not support having a query key in the availability call and, even for the ones that support it (as UNICEF e.g.), it is still wrong if the query key does not change. A new call should be fired only the first time or when the query key changes.

We'll improve this in next release.

amattioc avatar Dec 09 '25 14:12 amattioc