metafacture-core

metafacture-core copied to clipboard

Add JSON schema validator

3

Originally developed in OERSI. Will resolve #443.

fsteeg

Add SitemapReader originally developed in OERSI

2

Reads sitemap from URL, sends each `loc` URL to the receiver. e.g. `"https://hoou.de/sitemap.xml" | read-sitemap | open-http ...` in a Flux workflow to process every document linked in the sitemap....

fsteeg

Reported at https://github.com/hbz/lobid-resources/issues/1030#issue-502574421: When more threads are used than documents exists a `NullPointerException` occurs. When calling `closeStream()`, a `NullPointerException` appears. This stops the ETL process finishing correctly and results in...

dr0i

Counted leader elements in marc when encoding to marc

3

@maipet hinted that encode-marc21 or encode-marcxml cannot create the leader correctly since the elements are not counted. Could you elaborate the problem

TobiasNx

Provide namespace for xml-file in flux

5

Even though following ticket states that this is solved, https://github.com/metafacture/metafacture-core/issues/247 One cannot add a simple namespace to an xml file with flux or its java implementation. This should be easily...

TobiasNx

decode-csv struggles with escape sign "\" in string-values

5

https://metafacture.org/playground/?flux=inputFile%0A%7C+open-file%0A%7Cas-lines%0A%7Cdecode-csv%0A%7Cencode-json%0A%7Cprint%0A%3B&data=%22a%22%2C%22b\t%22%2C%22c\\t%22%2C%22\%22%2C%22\cd\%22 in: ```CSV "a","b\t","c\\t","\","\cd\" ``` FLUX: ```C open-file|as-lines|decode-csv|encode-json|print; ``` out: ```JSON {"0":"a","1":"bt","2":"c\\t","3":"\",cd\""} ``` expected: ```JSON {"0":"a","1":"b\\t","2":"c\\\t","3":"\\","4":"\\cd\\"} ```

TobiasNx

Similar to OAI PMH opener. Should retriev all records of a SRU request. Should be configurable with the specific SRU keywords.

TobiasNx

Decoder usually are turning incoming data into evenstream or xmlstream, but `decode-string` turns a string into a string. This is confusing. I suggest changing this name into `split-string` and keep...

TobiasNx

Working on a LIDO document an unconfigured `handle-generic-xml` flux command doesn't yield anything. This could somehow be configured to be logged ? -The user should be hinted to configure "recordtagname"....

dr0i

Update MarcXmlHandler.java

2

Update documentation for namespace handling in handle-marcxml. See: https://github.com/hbz/lobid-resources/issues/1348#issuecomment-1160495067

TobiasNx