dataverse icon indicating copy to clipboard operation
dataverse copied to clipboard
verified publisher icon IQSS

Expose export formats in native API

Open julian-schneider opened this issue 1 year ago • 7 comments

What this PR does / why we need it: The PR adds a method that exposes the full list of available exporters (display- and format names). This is especially useful for instances that have added custom formats.

Which issue(s) this PR closes: I know no pre-existing issues for this.

Suggestions on how to test this: New method found here: /api/info/exportFormats

Does this PR introduce a user interface change? If mockups are available, please link/include them here: No, just API and documentation changes.

API answer looks like this:

{
  "status": "OK",
  "data": [
    {
      "displayName": "OAI_ORE",
      "formatName": "OAI_ORE"
    },
    {
      "displayName": "DataCite",
      "formatName": "Datacite"
    },
    {
      "displayName": "Dublin Core",
      "formatName": "oai_dc"
    },
    {
      "displayName": "OpenAIRE",
      "formatName": "oai_datacite"
    },
    {
      "displayName": "Schema.org JSON-LD",
      "formatName": "schema.org"
    },
    {
      "displayName": "DDI",
      "formatName": "ddi"
    },
    {
      "displayName": "Dublin Core",
      "formatName": "dcterms"
    },
    {
      "displayName": "DDI HTML Codebook",
      "formatName": "html"
    },
    {
      "displayName": "JSON",
      "formatName": "dataverse_json"
    },
    {
      "displayName": "DDI",
      "formatName": "oai_ddi"
    }
  ]
}

Rendered docs entry looks like this: image

Is there a release notes update needed for this change?: New API method for listing the available exporters. Found at /api/info/exportFormats, produces display names and format names.

julian-schneider avatar Aug 02 '24 12:08 julian-schneider

Thanks for your feedback @pdurbin - I will be absent for two weeks, so some time will pass before I revise this. See you then!

julian-schneider avatar Aug 02 '24 14:08 julian-schneider

The response is now an object, and contains more properties per format, as suggested.

New response looks like this:

(expand response) 
{
  "status": "OK",
  "data": {
    "OAI_ORE": {
      "displayName": "OAI_ORE",
      "mediaType": "application/json",
      "isHarvestable": false,
      "isVisibleInUserInterface": true
    },
    "Datacite": {
      "displayName": "DataCite",
      "mediaType": "application/xml",
      "isHarvestable": true,
      "isVisibleInUserInterface": true,
      "XMLNameSpace": "http://datacite.org/schema/kernel-3",
      "XMLSchemaLocation": "http://datacite.org/schema/kernel-3 http://schema.datacite.org/meta/kernel-3/metadata.xsd",
      "XMLSchemaVersion": "3.0"
    },
    "oai_dc": {
      "displayName": "Dublin Core",
      "mediaType": "application/xml",
      "isHarvestable": true,
      "isVisibleInUserInterface": false,
      "XMLNameSpace": "http://www.openarchives.org/OAI/2.0/oai_dc/",
      "XMLSchemaLocation": "http://www.openarchives.org/OAI/2.0/oai_dc.xsd",
      "XMLSchemaVersion": "2.0"
    },
    "oai_datacite": {
      "displayName": "OpenAIRE",
      "mediaType": "application/xml",
      "isHarvestable": true,
      "isVisibleInUserInterface": true,
      "XMLNameSpace": "http://datacite.org/schema/kernel-4",
      "XMLSchemaLocation": "http://schema.datacite.org/meta/kernel-4.1/metadata.xsd",
      "XMLSchemaVersion": "4.1"
    },
    "schema.org": {
      "displayName": "Schema.org JSON-LD",
      "mediaType": "application/json",
      "isHarvestable": false,
      "isVisibleInUserInterface": true
    },
    "ddi": {
      "displayName": "DDI",
      "mediaType": "application/xml",
      "isHarvestable": false,
      "isVisibleInUserInterface": true,
      "XMLNameSpace": "ddi:codebook:2_5",
      "XMLSchemaLocation": "https://ddialliance.org/Specification/DDI-Codebook/2.5/XMLSchema/codebook.xsd",
      "XMLSchemaVersion": "2.5"
    },
    "dcterms": {
      "displayName": "Dublin Core",
      "mediaType": "application/xml",
      "isHarvestable": false,
      "isVisibleInUserInterface": true,
      "XMLNameSpace": "http://purl.org/dc/terms/",
      "XMLSchemaLocation": "http://dublincore.org/schemas/xmls/qdc/dcterms.xsd",
      "XMLSchemaVersion": "2.0"
    },
    "html": {
      "displayName": "DDI HTML Codebook",
      "mediaType": "text/html",
      "isHarvestable": false,
      "isVisibleInUserInterface": true
    },
    "dataverse_json": {
      "displayName": "JSON",
      "mediaType": "application/json",
      "isHarvestable": true,
      "isVisibleInUserInterface": true
    },
    "oai_ddi": {
      "displayName": "DDI",
      "mediaType": "application/xml",
      "isHarvestable": true,
      "isVisibleInUserInterface": false,
      "XMLNameSpace": "ddi:codebook:2_5",
      "XMLSchemaLocation": "https://ddialliance.org/Specification/DDI-Codebook/2.5/XMLSchema/codebook.xsd",
      "XMLSchemaVersion": "2.5"
    }
  }
}
New docs look like this:

Screenshot from 2024-08-28 11-05-16

julian-schneider avatar Aug 28 '24 09:08 julian-schneider

@pdurbin I believe I have addressed all the feedback you gave, have a nice day!

julian-schneider avatar Aug 28 '24 12:08 julian-schneider

Coverage Status

coverage: 20.76% (+0.02%) from 20.741% when pulling 3f81981287643f2ac7d37bce8f29bc86f9088ca6 on julian-schneider:expose-export-formats into 0d279573bb8d7c96d7a4a1dc4b66b2258059dfba on IQSS:develop.

coveralls avatar Aug 31 '24 11:08 coveralls

@pdurbin @GPortas could be get this API into the next release? This is a rather simple feature needed for the SPA to display the list of export formats.

johannes-darms avatar Oct 11 '24 09:10 johannes-darms

@johannes-darms I do like the PR a lot. @cmbz @scolapasta what do you think? Can we put a 6.5 milestone on it or at least put it in "sprint ready"?

Also, I agree with Johannes that the SPA will need this some day. We might want to see what @ekraffmiller @ChengShi-1 @GPortas and @g-saracca think about the output, if they are happy with it.

pdurbin avatar Oct 15 '24 18:10 pdurbin

2024/10/15: Added to sprint ready after conversation with @pdurbin

cmbz avatar Oct 15 '24 18:10 cmbz

@julian-schneider tests are failing in InfoIT.testGetExportFormats. Can you please take a look?

Error
data.Datacite.XMLNameSpace
Expected: http://datacite.org/schema/kernel-3
     got: http://datacite.org/schema/kernel-4
 ; data.Datacite.XMLSchemaLocation
Expected: http://datacite.org/schema/kernel-3 http://schema.datacite.org/meta/kernel-3/metadata.xsd
     got: http://datacite.org/schema/kernel-4 http://schema.datacite.org/meta/kernel-4.5/metadata.xsd
 ; data.Datacite.XMLSchemaVersion
Expected: 3.0
     got: 4.5
 ; data.ddi.displayName
Expected: DDI
     got: DDI Codebook v2
 ; data.oai_ddi.displayName
Expected: DDI
     got: DDI Codebook v2
Stacktrace
java.lang.AssertionError: 
data.Datacite.XMLNameSpace
Expected: http://datacite.org/schema/kernel-3
     got: http://datacite.org/schema/kernel-4
 ; data.Datacite.XMLSchemaLocation
Expected: http://datacite.org/schema/kernel-3 http://schema.datacite.org/meta/kernel-3/metadata.xsd
     got: http://datacite.org/schema/kernel-4 http://schema.datacite.org/meta/kernel-4.5/metadata.xsd
 ; data.Datacite.XMLSchemaVersion
Expected: 3.0
     got: 4.5
 ; data.ddi.displayName
Expected: DDI
     got: DDI Codebook v2
 ; data.oai_ddi.displayName
Expected: DDI
     got: DDI Codebook v2
	at org.skyscreamer.jsonassert.JSONAssert.assertEquals(JSONAssert.java:417)
	at org.skyscreamer.jsonassert.JSONAssert.assertEquals(JSONAssert.java:394)
	at org.skyscreamer.jsonassert.JSONAssert.assertEquals(JSONAssert.java:336)
	at edu.harvard.iq.dataverse.api.InfoIT.testGetExportFormats(InfoIT.java:99)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Standard Output
{
    "status": "OK",
    "data": {
        "OAI_ORE": {
            "displayName": "OAI_ORE",
            "mediaType": "application/json",
            "isHarvestable": false,
            "isVisibleInUserInterface": true
        },
        "Datacite": {
            "displayName": "DataCite",
            "mediaType": "application/xml",
            "isHarvestable": true,
            "isVisibleInUserInterface": true,
            "XMLNameSpace": "http://datacite.org/schema/kernel-4",
            "XMLSchemaLocation": "http://datacite.org/schema/kernel-4 http://schema.datacite.org/meta/kernel-4.5/metadata.xsd",
            "XMLSchemaVersion": "4.5"
        },
        "oai_dc": {
            "displayName": "Dublin Core",
            "mediaType": "application/xml",
            "isHarvestable": true,
            "isVisibleInUserInterface": false,
            "XMLNameSpace": "http://www.openarchives.org/OAI/2.0/oai_dc/",
            "XMLSchemaLocation": "http://www.openarchives.org/OAI/2.0/oai_dc.xsd",
            "XMLSchemaVersion": "2.0"
        },
        "oai_datacite": {
            "displayName": "OpenAIRE",
            "mediaType": "application/xml",
            "isHarvestable": true,
            "isVisibleInUserInterface": true,
            "XMLNameSpace": "http://datacite.org/schema/kernel-4",
            "XMLSchemaLocation": "http://schema.datacite.org/meta/kernel-4.1/metadata.xsd",
            "XMLSchemaVersion": "4.1"
        },
        "schema.org": {
            "displayName": "Schema.org JSON-LD",
            "mediaType": "application/json",
            "isHarvestable": false,
            "isVisibleInUserInterface": true
        },
        "ddi": {
            "displayName": "DDI Codebook v2",
            "mediaType": "application/xml",
            "isHarvestable": false,
            "isVisibleInUserInterface": true,
            "XMLNameSpace": "ddi:codebook:2_5",
            "XMLSchemaLocation": "https://ddialliance.org/Specification/DDI-Codebook/2.5/XMLSchema/codebook.xsd",
            "XMLSchemaVersion": "2.5"
        },
        "dcterms": {
            "displayName": "Dublin Core",
            "mediaType": "application/xml",
            "isHarvestable": false,
            "isVisibleInUserInterface": true,
            "XMLNameSpace": "http://purl.org/dc/terms/",
            "XMLSchemaLocation": "http://dublincore.org/schemas/xmls/qdc/dcterms.xsd",
            "XMLSchemaVersion": "2.0"
        },
        "html": {
            "displayName": "DDI HTML Codebook",
            "mediaType": "text/html",
            "isHarvestable": false,
            "isVisibleInUserInterface": true
        },
        "dataverse_json": {
            "displayName": "JSON",
            "mediaType": "application/json",
            "isHarvestable": true,
            "isVisibleInUserInterface": true
        },
        "oai_ddi": {
            "displayName": "DDI Codebook v2",
            "mediaType": "application/xml",
            "isHarvestable": true,
            "isVisibleInUserInterface": false,
            "XMLNameSpace": "ddi:codebook:2_5",
            "XMLSchemaLocation": "https://ddialliance.org/Specification/DDI-Codebook/2.5/XMLSchema/codebook.xsd",
            "XMLSchemaVersion": "2.5"
        }
    }
}

pdurbin avatar Nov 20 '24 13:11 pdurbin

No issues with PR discovered. Additional tests were done in internal before and after installing Croissant. As expected, Croissant did appear in /api/info/exportFormats after installation Testing of 10739.docx

ofahimIQSS avatar Nov 22 '24 18:11 ofahimIQSS