Marek Horst
Marek Horst
Consider removing poperties values from workflow.xml which are already defined in default-config.xml
Some properties (e.g. `*SparkExecutorOverhead`) values apart from being declared in `default-config.xml` file (which is part of the environment, originally defined in [this config git repo](https://git.icm.edu.pl/openaire/iis-deployment/-/blob/master/deploy/apps/primary/wf/primary/main/config-default/config-default.template)) are also explicitly declared in...
Currently the working SH db file location: ``` /cache/external-resources/software-heritage/origins.db ``` which is used by the IIS is also dropped before new DB version is generated. When the process fails then...
It looks like all implementations of `AbstractDBBuilder#initializeProcess()` in the following subclasses: * `CommunityDBBuilder` * `DatasetDBBuilder` * `ServiceDBBuilder` * `PatentDBBuilder` * `ProjectDBBuilder` * `ResearchInitiativeDBBuilder` * `SoftwareHeritageOriginDBBuilder` are quite similar and it...
Originally requested in: https://support.openaire.eu/issues/6717#note-35 Integrate https://github.com/openaire/iis/pull/1387 PR and align the integration tests whenever needed.
Currently `avro2json` workflow is responsible for translating avro input into json output. It will be convenient, also for #918, to allow enabling compression by providing e.g. `compression_method` input parameter which...
After `DeduplicationMapping` -> `IdentifierMapping` refactoring done in #1264 we should get rid of `DeduplicationMappingConverter` class and all the references classes. `DeduplicationMappingConverterTest` should also be removed.
In the #1361 pull request we have replaced the ObjectStore as the source of contents metadata with the hive-based aggregation subsystem. The identifier matching strategy was simplified, as described in...
Originally reported by @mpol in https://github.com/openaire/iis/pull/1361#discussion_r909599096. I guess it might be some leftover when the IIS test code base was switched from junit4 to junit5 as it seems the `@BeforeEach`...
After integrating the https://github.com/openaire/iis/blob/marekhorst_1298_refactor_iis_content_importer branch with https://github.com/openaire/iis/pull/1361 pull request we end up with one `eu.dnetlib.iis.wf.importer.content.WorkflowTest#testImportContentUrlWorkflowWithHiveBasedAggregationSubsystemAsBackend()` test as disabled because it can't be run on CI test cluster due to missing...
#1361 pull request (closing the #1298 issue) introduced the new aggregation subsystem based content importer module while still supporting an old, ObjectStore based, approach which is meant to be considered...