metafacture-core

metafacture-core copied to clipboard

Release Metafacture core 6.1.1

2

This release fixes bugs introduced in 6.1.0 (https://github.com/metafacture/metafacture-core/issues/527#issuecomment-2188447331). All new features since 6.0.0 will be documented in 6.1.1. - [x] update dependencies if these are promising to not break much...

dr0i

encode-marcxml should output leader as first element, all controlfield after that and then all datafields after that

8

With the last release `encode-marcxml` outputs the leader as last element, it should be the first. See: https://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd ``` ```

TobiasNx

batch-reset creates an empty record at the end when writing and many empty records with encode marcxml

1

See: https://github.com/TobiasNx/metafacture_workflows/commit/64ba3e8fdb57c79c4888b4ec526f7aa4a427678c Running the workflow with Metafix-Runner 1.0.0

TobiasNx

Write valid leader without counting #527

3

Instead of just appending the incoming leader elements by `encode-marcmxl`, this adds the static values for Pos: 00-04, 10-16, 20-23. By using `0`as default value for Pos: 00-04 and 12-16....

TobiasNx

Make namespace check for decode-xml optional

4

https://metafacture.org/playground/?flux=inputFile%0A%7C+open-file%0A%7Cdecode-xml%0A%7Chandle-generic-xml%0A%7Cencode-yaml%0A%7Cprint%0A%3B&data=%3C%3Fxml+version%3D%221.0%22%3F%3E%0A%3Crecord%3E%0A++++%3Cmets%3Afield%3Ea%3C/mets%3Afield%3E%0A++++%3Cfield%3Eb%3C/field%3E%0A%3C/record%3E When reading a xml with namespaces but no corresponding namespace definition MF breaks. This should be optional.

TobiasNx

`handle-generic-xml` roottag option does not work with namespace-prefix

1

`|handle-generic-xml(emitnamespace="true",attributemarker="@",recordtagname="mets:mets")` does not work: https://metafacture.org/playground/?flux=%22https%3A//raw.githubusercontent.com/TobiasNx/metafacture_workflows/b2f8776294ee50aa2e14a43fc2a08d6bfdc81775/replaceXSLTTransformation/xsltOutput/UBi_01_Beispiel_Ergebnis.xml%22%0A%7Copen-http%0A%7Cdecode-xml%0A%7Chandle-generic-xml%28emitnamespace%3D%22true%22%2Cattributemarker%3D%22@%22%2Crecordtagname%3D%22mets%3Amets%22%29%0A%7Cencode-json%28prettyPrinting%3D%22true%22%29%0A%7Cprint%0A%3B while |handle-generic-xml(emitnamespace="true",attributemarker="@",recordtagname="mets") works: https://metafacture.org/playground/?flux=%22https%3A//raw.githubusercontent.com/TobiasNx/metafacture_workflows/b2f8776294ee50aa2e14a43fc2a08d6bfdc81775/replaceXSLTTransformation/xsltOutput/UBi_01_Beispiel_Ergebnis.xml%22%0A%7Copen-http%0A%7Cdecode-xml%0A%7Chandle-generic-xml%28emitnamespace%3D%22true%22%2Cattributemarker%3D%22@%22%2Crecordtagname%3D%22mets%22%29%0A%7Cencode-json%28prettyPrinting%3D%22true%22%29%0A%7Cprint%0A%3B Not sure how problematic this is, but at least it should be documented.

TobiasNx

Namespace in encode-xml changes order

2

The order of the namespace input is not the same as given in the option of the flux modul: https://metafacture.org/playground/?flux=inputFile%0A%7Copen-file%0A%7Cas-lines%0A%7Cdecode-json%0A%7Cfix%28transformationFile%29%0A%7Cencode-xml%28recordtag%3D%22%22%2Croottag%3D%22mets%3Amets%22%2Cattributemarker%3D%22@%22%2Cvaluetag%3D%22value%22%2C+namespaces%3D%22__default%3Dhttp%3A//www.w3.org/TR/html4/\ndcterms%3Dhttp%3A//purl.org/dc/terms/creator\ndv%3Dhttp%3A//dfg-viewer.de/\nvl%3Dhttp%3A//visuallibrary.net/vl\nvls%3Dhttp%3A//semantics.de/vls\nmarcxml%3Dhttp%3A//www.loc.gov/MARC21/slim\nepicur%3Durn%3Anbn%3Ade%3A1111-2004033116\noai%3Dhttp%3A//www.openarchives.org/OAI/2.0/\nzvdd%3Dhttp%3A//zvdd.gdz-cms.de/\nmets%3Dhttp%3A//www.loc.gov/METS/\nvlz%3Dhttp%3A//visuallibrary.net/vlz/1.0\nmods%3Dhttp%3A//www.loc.gov/mods/v3\nxlink%3Dhttp%3A//www.w3.org/1999/xlink\nxmlns%3Amix%3Dhttp%3A//www.loc.gov/mix/v20\ndc%3Dhttp%3A//purl.org/dc/elements/1.1/%22%29%0A%7Cprint%0A%3B&transformation=move_field%28%22_id%22%2C+%22id%22%29%0Amove_field%28%22a%22%2C+%22title%22%29%0Apaste%28%22creator%22%2C+%22b.v%22%2C+%22b.n%22%2C+%22~aus%22%2C+%22c%22%29%0Aretain%28%22id%22%2C+%22title%22%2C+%22creator%22%29%0A%0A&data=%7B%22a%22%3A+%22Faust%22%2C+%22b%22%3A+%7B%22n%22%3A+%22Goethe%22%2C+%22v%22%3A+%22JW%22%7D%2C+%22c%22%3A+%22Weimar%22%7D%0A%7B%22a%22%3A+%22R%C3%A4uber%22%2C+%22b%22%3A+%7B%22n%22%3A+%22Schiller%22%2C+%22v%22%3A+%22F%22%7D%2C+%22c%22%3A+%22Weimar%22%7D

TobiasNx

Make encoder-marc21 more forgiving?

6

Came up in https://github.com/metafacture/metafacture-core/issues/527 : If we [parse (assumingly) crude binary `MARC` the encoding fails](https://metafacture.org/playground/?flux=%22https%3A//raw.githubusercontent.com/gbv/Catmandu-Tutorial/master/data/marc.mrc%22%0A%7C+open-http%0A%7C+as-lines%0A%7C+decode-marc21%28emitleaderaswhole%3D%22true%22%29%0A%7C+encode-marc21%0A%7C+print%0A%3B). (first broken `MRC` seems to be `02589nas a2200601 c 4500` in https://raw.githubusercontent.com/gbv/Catmandu-Tutorial/master/data/marc.mrc (should be double...

dr0i

Add command `string-to-variable` to reuse incoming string as variable

9

At the moment we cannot use the incoming url-string after it is used in `open-http`. A useful scenario would be if we scrape a website but the website does not...

TobiasNx

`decode-xml` throws exception due to a missing DTD File on the web. [See here](https://metafacture.org/playground/?flux=inputFile%0A%7Copen-file%0A%7Cas-records%0A%7Cread-string+%0A%7Cdecode-xml%0A%7Chandle-generic-xml%28recordtagname%3D%22ONIXmessage%22%29%0A%7Cencode-literals%0A%7Cprint%0A%3B&data=%3C%3Fxml+version%3D%271.0%27+encoding%3D%27UTF-8%27%3F%3E%0A%3C%21DOCTYPE+ONIXmessage+SYSTEM%0A%22http%3A//www.editeur.org/onix/2.1/short/onix-international.dtd%22%3E%0A%3CONIXmessage+release%3D%222.1%22%3E%0A%09%3Cheader%3E%0A%09%09%3Cm174%3ELogos+Verlag+Berlin%3C/m174%3E%0A%09%09%3Cm175%3EDr.+Volkhard+Buchholtz%3C/m175%3E%0A%09%09%3Cm283%[email protected]%3C/m283%3E%0A%09%09%3Cm182%3E20240208%3C/m182%3E%0A%09%3C/header%3E%0A%09%3Cproduct%3E%0A%09%09%3Ca001%3E9783832545482%3C/a001%3E%0A%09%09%3Ca002%3E03%3C/a002%3E%0A%09%09%3Cproductidentifier%3E%0A%09%09%09%3Cb221%3E03%3C/b221%3E%0A%09%09%09%3Cb244%3E9783832545482%3C/b244%3E%0A%09%09%3C/productidentifier%3E%0A%09%09%3Cproductidentifier%3E%0A%09%09%09%3Cb221%3E06%3C/b221%3E%0A%09%09%09%3Cb244%3E10.30819/4548%3C/b244%3E%0A%09%09%3C/productidentifier%3E%0A%09%09%3Cb012%3EDG%3C/b012%3E%0A%09%09%3Cb211%3E002%3C/b211%3E%0A%09%09%3Cseries%3E%0A%09%09%09%3Ctitle%3E%0A%09%09%09%09%3Cb202%3E01%3C/b202%3E%0A%09%09%09%09%3Cb203%3EPhilosophische+Hefte%3C/b203%3E%0A%09%09%09%3C/title%3E%0A%09%09%09%3Cb019%3E1%3C/b019%3E%0A%09%09%3C/series%3E%0A%09%09%3Ctitle%3E%0A%09%09%09%3Cb202%3E01%3C/b202%3E%0A%09%09%09%3Cb203%3EUngewissheit+und+Objektivit%C3%A4t%3C/b203%3E%0A%09%09%3C/title%3E%0A%09%09%3Ccontributor%3E%0A%09%09%09%3Cb034%3E1%3C/b034%3E%0A%09%09%09%3Cb035%3EA01%3C/b035%3E%0A%09%09%09%3Cb036%3EThomas+Gil%3C/b036%3E%0A%09%09%09%3Cb039%3EThomas%3C/b039%3E%0A%09%09%09%3Cb040%3EGil%3C/b040%3E%0A%09%09%3C/contributor%3E%0A%09%09%3Clanguage%3E%0A%09%09%09%3Cb253%3E01%3C/b253%3E%0A%09%09%09%3Cb252%3Eger%3C/b252%3E%0A%09%09%3C/language%3E%0A%09%09%3Cb061%3E43%3C/b061%3E%0A%09%09%3Cmainsubject%3E%0A%09%09%09%3Cb191%3E26%3C/b191%3E%0A%09%09%09%3Cb068%3E2.0%3C/b068%3E%0A%09%09%09%3Cb069%3E1922%3C/b069%3E%0A%09%09%3C/mainsubject%3E%0A%09%09%3Cmainsubject%3E%0A%09%09%09%3Cb191%3E93%3C/b191%3E%0A%09%09%09%3Cb068%3E1.0%3C/b068%3E%0A%09%09%09%3Cb069%3EQDHR%3C/b069%3E%0A%09%09%3C/mainsubject%3E%0A%09%09%3Csubject%3E%0A%09%09%09%3Cb067%3E93%3C/b067%3E%0A%09%09%09%3Cb068%3E1.0%3C/b068%3E%0A%09%09%09%3Cb069%3EGTB%3C/b069%3E%0A%09%09%3C/subject%3E%0A%09%09%3Csubject%3E%0A%09%09%09%3Cb067%3E93%3C/b067%3E%0A%09%09%09%3Cb068%3E1.0%3C/b068%3E%0A%09%09%09%3Cb069%3EQDHR9%3C/b069%3E%0A%09%09%3C/subject%3E%0A%09%09%3Csubject%3E%0A%09%09%09%3Cb067%3E93%3C/b067%3E%0A%09%09%09%3Cb068%3E1.0%3C/b068%3E%0A%09%09%09%3Cb069%3EYPJL%3C/b069%3E%0A%09%09%3C/subject%3E%0A%09%09%3Csubject%3E%0A%09%09%09%3Cb067%3E20%3C/b067%3E%0A%09%09%09%3Cb070%3EInduktion+und+Induktives+Schlie%C3%9Fen%3C/b070%3E%0A%09%09%3C/subject%3E%0A%09%09%3Csubject%3E%0A%09%09%09%3Cb067%3E20%3C/b067%3E%0A%09%09%09%3Cb070%3EKonditionalisiertes+Denken+und+bedingte+Wahrscheinlichkeiten%3C/b070%3E%0A%09%09%3C/subject%3E%0A%09%09%3Csubject%3E%0A%09%09%09%3Cb067%3E20%3C/b067%3E%0A%09%09%09%3Cb070%3EObjektivit%C3%A4t%3C/b070%3E%0A%09%09%3C/subject%3E%0A%09%09%3Csubject%3E%0A%09%09%09%3Cb067%3E20%3C/b067%3E%0A%09%09%09%3Cb070%3EModelle+und+inferentielle+Netzwerke%3C/b070%3E%0A%09%09%3C/subject%3E%0A%09%09%3Csubject%3E%0A%09%09%09%3Cb067%3E20%3C/b067%3E%0A%09%09%09%3Cb070%3EAnalogiebildungen+und+nicht-demonstratives+Schlie%C3%9Fen%3C/b070%3E%0A%09%09%3C/subject%3E%0A%09%09%3Cothertext%3E%0A%09%09%09%3Cd102%3E01%3C/d102%3E%0A%09%09%09%3Cd103%3E00%3C/d103%3E%0A%09%09%09%3Cd104%3E%0A%09%09%09%09Aus+Erfahrenem+schlie%C3%9Fen+wir%2C+wie+die+Dinge+der+Welt+beschaffen+sind+und+was+wir+in+Bezug%0A%09%09%09%09auf+die+Zukunft+erwarten+k%C3%B6nnen.+Unser+Schlie%C3%9Fen+wird+aber+immer+ungewiss+bleiben%2C+selbst%0A%09%09%09%09dann%2C+wenn+wir+uns+um+Objektivit%C3%A4t+bem%C3%BChen.%0A%0A%09%09%09%3C/d104%3E%0A%09%09%3C/othertext%3E%0A%09%09%3Cothertext%3E%0A%09%09%09%3Cd102%3E47%3C/d102%3E%0A%09%09%09%3Cd103%3E06%3C/d103%3E%0A%09%09%09%3Cd104%3EOpen+Access+publication%3C/d104%3E%0A%09%09%3C/othertext%3E%0A%09%09%3Cothertext%3E%0A%09%09%09%3Cd102%3E46%3C/d102%3E%0A%09%09%09%3Cd105%3E01%3C/d105%3E%0A%09%09%09%3Cd106%3Ehttp%3A//creativecommons.org/licenses//4.0%3C/d106%3E%0A%09%09%3C/othertext%3E%0A%09%09%3Cproductwebsite%3E%0A%09%09%09%3Cb367%3E02%3C/b367%3E%0A%09%09%09%3Cf123%3Ehttps%3A//doi.org/10.30819/4548%3C/f123%3E%0A%09%09%3C/productwebsite%3E%0A%09%09%3Cpublisher%3E%0A%09%09%09%3Cb081%3ELogos+Verlag+Berlin%3C/b081%3E%0A%09%09%3C/publisher%3E%0A%09%09%3Cb209%3EBerlin%3C/b209%3E%0A%09%09%3Cb003%3E20171013%3C/b003%3E%0A%09%3C/product%3E%0A%3C/ONIXmessage%3E) The error is the Doctype declaration that is not available anymore on the web. [See workaround...

TobiasNx